Hyper camera with shared mirror

ABSTRACT

This disclosure is related to positioning one or more glass plates between an image sensor and lens of a camera in a scanning camera system; determining plate rotation rates and plate rotation angles based on one of characteristics of the camera, characteristics and positioning of the one or more glass plates, and relative dynamics of the camera and the object area; and rotating the one or more glass plates about one or more predetermined axes based on corresponding plate rotation rates and plate rotation angles.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to PCT Application No.PCT/US21/39333, filed Jun. 28, 2021, the contents of which areincorporated by reference herein in their entirety for all purposes.

FIELD OF THE INVENTION

The present invention relates to efficient aerial camera systems andefficient methods for creating orthomosaics and textured 3D models fromaerial photos.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent the work is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Accurately georeferenced mosaics of orthophotos, referred to asorthomosaics, can be created from aerial photos. In such a case, thesephotos can provide useful images of an area, such as the ground. Thecreation of an orthomosaic requires the systematic capture ofoverlapping aerial photos of the region of interest (ROI), both toensure complete coverage of the ROI, and to ensure that there issufficient redundancy in the imagery to allow accurate bundleadjustment, orthorectification and alignment of the photos.

Bundle adjustment is the process by which redundant estimates of groundpoints and camera poses are refined. Bundle adjustment may operate onthe positions of manually-identified ground points, or, increasingly, onthe positions of automatically-identified ground features which areautomatically matched between overlapping photos.

Overlapping aerial photos are typically captured by navigating a surveyaircraft in a serpentine pattern over the area of interest. The surveyaircraft carries an aerial scanning camera system, and the serpentineflight pattern ensures that the photos captured by the scanning camerasystem overlap both along flight lines within the flight pattern andbetween adjacent flight lines.

Though such scanning camera systems can be useful in some instances,they are not without their flaws. Examples of such flaws include: (1)difficulty fitting several long focal length lenses and matched aperturemirrors in configured spaces on a vehicle for capturing vertical andoblique imagery; (2) a camera hole in an aerial vehicle is generallyrectangular, but yaw correction gimbal space requirements are defined bya circle, so inefficiencies in spacing are present; and (3) low qualityimages (e.g. blurry, vignetting).

SUMMARY

The present disclosure is directed to an imaging system, comprising: acamera configured to capture an image of an object area from an imagingbeam from the object area, the camera including an image sensor and alens; one or more glass plates positioned between the image sensor andthe lens of the camera; one or more first drives coupled to each of theone or more glass plates; a scanning mirror structure including at leastone mirror surface; a second drive coupled to the scanning mirrorstructure and configured to rotate the scanning mirror structure about ascan axis based on a scan angle; and a motion compensation systemconfigured to determine at least one of plate rotation rates and platerotation angles based on relative dynamics of the imaging system and theobject area and optical properties of the one or more glass plates; andcontrol the one or more first drives to rotate the one or more glassplates about one or more predetermined axes based on at least one ofcorresponding plate rotation rates and plate rotation angles.

The present disclosure is directed to an imaging method, comprising:reflecting an imaging beam from an object area using at least one mirrorsurface of a scanning mirror structure to an image sensor of a camera tocapture a set of images along a scan path of the object area, the cameracomprising a lens and an image sensor; capturing an image from theimaging beam from the object area reflected by the at least one mirrorsurface using the image sensor of the camera; positioning one or moreglass plates between the image sensor and the lens of the camera;determining plate rotation rates and plate rotation angles based on oneof characteristics of the camera, characteristics and positioning of theone or more glass plates, and relative dynamics of the camera and theobject area; and rotating the one or more glass plates about one or morepredetermined axes based on corresponding plate rotation rates and platerotation angles.

BRIEF DESCRIPTION OF FIGURES

A more complete understanding of this disclosure is provided byreference to the following detailed description when considered inconnection with the accompanying drawings, wherein:

FIG. 1 a shows scan patterns for a scanning camera system taken from astationary aerial vehicle, according to one exemplary embodiment of thepresent disclosure;

FIG. 1 b shows overlapping sets of scan patterns for a scanning camerasystem taken from a stationary aerial vehicle, according to oneexemplary embodiment of the present disclosure;

FIG. 2 shows a serpentine flight path that an aerial vehicle can take tocapture images using a scanning camera system, according to oneexemplary embodiment of the present disclosure;

FIG. 3 shows distribution views at various ground locations for ascanning camera system, according to one exemplary embodiment of thepresent disclosure;

FIG. 4 a shows a scan drive unit from a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 4 b shows the scan drive unit from a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 4 c shows a scan pattern captured by the scan drive unit from a topdown view, according to one exemplary embodiment of the presentdisclosure;

FIG. 4 d shows the scan pattern captured by the scan drive unit from anoblique view, according to one exemplary embodiment of the presentdisclosure;

FIG. 4 e shows a first set of potential geometries for a scanning mirrorstructure in the scan drive unit, according to one exemplary embodimentof the present disclosure;

FIG. 4 f shows a second set of potential geometries for the scanningmirror structure in the scan drive unit, according to one exemplaryembodiment of the present disclosure;

FIG. 4 g shows potential geometries for scanning mirror structures andpaddle flaps, according to one exemplary embodiment of the presentdisclosure;

FIG. 5 a shows another scan drive unit from a first perspective,according to one exemplary embodiment of the present disclosure;

FIG. 5 b shows the scan drive unit from a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 5 c shows a scan pattern captured by the scan drive unit from a topdown view, according to one exemplary embodiment of the presentdisclosure;

FIG. 5 d shows the scan pattern captured by the scan drive unit from anoblique view, according to one exemplary embodiment of the presentdisclosure;

FIG. 5 e shows potential geometries for a primary mirror in the scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 5 f shows potential geometries for a secondary mirror in the scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 6 a shows another scan drive unit from a first perspective,according to one exemplary embodiment of the present disclosure;

FIG. 6 b shows the scan drive unit from a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 6 c shows a scan pattern captured by the scan drive unit from a topdown view, according to one exemplary embodiment of the presentdisclosure;

FIG. 6 d shows the scan pattern captured by the scan drive unit from anoblique view, according to one exemplary embodiment of the presentdisclosure;

FIG. 6 e shows potential geometries for a primary mirror in the scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 6 f shows potential geometries for a secondary mirror in the scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 7 a shows a scanning camera system from a first perspective,according to one exemplary embodiment of the present disclosure;

FIG. 7 b shows the scanning camera system from a second perspective,according to one exemplary embodiment of the present disclosure;

FIG. 7 c shows the scanning camera system from a third perspective,according to one exemplary embodiment of the present disclosure;

FIG. 7 d shows the scanning camera system from a fourth perspective,according to one exemplary embodiment of the present disclosure;

FIG. 7 e shows scan patterns captured by the scanning camera system froma top down view, according to one exemplary embodiment of the presentdisclosure;

FIG. 7 f shows scan patterns captured by the scanning camera system froman oblique view, according to one exemplary embodiment of the presentdisclosure;

FIG. 8 a shows top down and oblique views of a scan pattern taken froman aerial vehicle with forward motion, according to one exemplaryembodiment of the present disclosure;

FIG. 8 b shows top down and oblique views of multiple sets of scanpatterns taken from an aerial vehicle with forward motion, according toone exemplary embodiment of the present disclosure;

FIG. 8 c shows top down and oblique views of multiple sets of scanpatterns, according to one exemplary embodiment of the presentdisclosure;

FIG. 9 shows a system diagram, according to one exemplary embodiment ofthe present disclosure;

FIG. 10 shows another system diagram, according to one exemplaryembodiment of the present disclosure;

FIG. 11 shows another system diagram, according to one exemplaryembodiment of the present disclosure;

FIG. 12 illustrates refraction of light at a glass plate, according toone exemplary embodiment of the present disclosure;

FIG. 13 a shows an arrangement for motion compensation in a camera of ascanning camera system from a perspective view, according to oneexemplary embodiment of the present disclosure;

FIG. 13 b shows the arrangement for motion compensation in the camera ofthe scanning camera system from a side view, according to one exemplaryembodiment of the present disclosure;

FIG. 13 c shows the arrangement for motion compensation in the camera ofthe scanning camera system from a view down the optical axis, accordingto one exemplary embodiment of the present disclosure;

FIG. 14 a shows another arrangement for motion compensation in a cameraof a scanning camera system from a perspective view, according to oneexemplary embodiment of the present disclosure;

FIG. 14 b shows the arrangement for motion compensation in the camera ofthe scanning camera system from a side view, according to one exemplaryembodiment of the present disclosure;

FIG. 14 c shows the arrangement for motion compensation in the camera ofthe scanning camera system from a view down the optical axis, accordingto one exemplary embodiment of the present disclosure;

FIG. 15 a shows another arrangement for motion compensation in a cameraof a scanning camera system from a perspective view, according to oneexemplary embodiment of the present disclosure;

FIG. 15 b shows the arrangement for motion compensation in the camera ofthe scanning camera system from a side view, according to one exemplaryembodiment of the present disclosure;

FIG. 15 c shows the arrangement for motion compensation in the camera ofthe scanning camera system from a view down the optical axis, accordingto one exemplary embodiment of the present disclosure;

FIG. 16 shows trajectories for tilt (top), tilt rate (middle), and tiltacceleration (bottom) for tilting plate motion, according to oneexemplary embodiment of the present disclosure;

FIG. 17 a shows various object area projection geometries andcorresponding sensor plots for motion compensation, according to oneexemplary embodiment of the present disclosure;

FIG. 17 b illustrates the motion compensation pixel velocity from FIG.17 a (upper) and corresponding tilt rates for a first and second opticalplate (lower), according to one exemplary embodiment of the presentdisclosure;

FIG. 18 a illustrates object area projection geometries andcorresponding sensor plots for motion compensation, according to oneexemplary embodiment of the present disclosure;

FIG. 18 b illustrates the motion compensation pixel velocity from FIG.18 a (upper) and corresponding plate rates for a first and secondoptical plate (lower), according to one exemplary embodiment of thepresent disclosure;

FIG. 19 a shows a tilt trajectory for the first optical plate from FIG.18 b that can be used to achieve motion compensation for the requiredtilt rate, according to one exemplary embodiment of the presentdisclosure;

FIG. 19 b show a tilt trajectory for the second optical plate from FIG.18 b that can be used to achieve motion compensation for the requiredtilt rate, according to one exemplary embodiment of the presentdisclosure;

FIG. 20 a illustrates pixel velocities and tilt rates for a first scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 20 b illustrates pixel velocities and tilt rates for a second scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 21 a illustrates pixel velocities and tilt rates for a first scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 21 b illustrates pixel velocities and tilt rates for a second scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 22 a illustrates pixel velocities and tilt rates for a first scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 22 b illustrates pixel velocities and tilt rates for a second scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 23 a illustrates pixel velocities and tilt rates for a first scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 23 b illustrates pixel velocities and tilt rates for a second scandrive unit, according to one exemplary embodiment of the presentdisclosure;

FIG. 24 shows a view of a scanning camera system; according to oneexemplary embodiment of the present disclosure;

FIG. 25 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole in the absence of roll, pitch or yaw,according to one exemplary embodiment of the present disclosure;

FIG. 26 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole with roll corrected using a stabilisationplatform, according to one exemplary embodiment of the presentdisclosure;

FIG. 27 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole with pitch corrected using astabilisation platform, according to one exemplary embodiment of thepresent disclosure;

FIG. 28 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole with yaw corrected using a stabilisationplatform, according to one exemplary embodiment of the presentdisclosure;

FIG. 29 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole where a stabilisation platform has notcorrected the yaw, according to one exemplary embodiment of the presentdisclosure;

FIG. 30 a shows top and oblique views of scan patterns for a scanningcamera system when the aerial vehicle has yaw, according to oneexemplary embodiment of the present disclosure;

FIG. 30 b shows top and oblique views of three sets of scan patternswith forward overlap for a scanning camera system when the aerialvehicle has yaw, according to one exemplary embodiment of the presentdisclosure;

FIG. 31 shows a top view (upper) and bottom view (lower) of a scanningcamera system in a survey hole for a case that the aerial vehicle hasyaw that has been corrected by an offset scan angle, according to oneexemplary embodiment of the present disclosure;

FIG. 32 a shows top and obliques views of scan patterns for a scanningcamera system when the aerial vehicle has yaw, according to oneexemplary embodiment of the present disclosure;

FIG. 32 b shows top and oblique views of three sets of scan patternswith forward overlap for a scanning camera system when the aerialvehicle has yaw, according to one exemplary embodiment of the presentdisclosure;

FIG. 33 a illustrates capturing an image without a ghost image beam,according to one exemplary embodiment of the present disclosure;

FIG. 33 b illustrates capturing an image with a ghost image beam,according to one exemplary embodiment of the present disclosure;

FIG. 34 a illustrates a hybrid mirror having low-reflectance material,according to one exemplary embodiment of the present disclosure;

FIG. 34 b illustrates using a hybrid mirror to prevent ghost images,according to one exemplary embodiment of the present disclosure;

FIG. 35 a illustrates vignetting caused by a survey hole, according toone exemplary embodiment of the present disclosure;

FIG. 35 b illustrates vignetting caused by a survey hole, according toone exemplary embodiment of the present disclosure;

FIG. 36 a shows an image of a uniform untextured surface affected byvignetting, according to one exemplary embodiment of the presentdisclosure;

FIG. 36 b illustrates vignetting at various locations on the image fromFIG. 36 a , according to one exemplary embodiment of the presentdisclosure;

FIG. 36 c shows an image obtained using a modified aperture and havingless vignetting, according to one exemplary embodiment of the presentdisclosure;

FIG. 36 d shows an example of regions that can define an aperture,according to one exemplary embodiment of the present disclosure;

FIG. 36 e shows an example of regions that can define an aperture,according to one exemplary embodiment of the present disclosure;

FIG. 36 f shows an example of regions that can define an aperture,according to one exemplary embodiment of the present disclosure;

FIG. 36 g shows an example of regions that can define an aperture,according to one exemplary embodiment of the present disclosure;

FIG. 36 h shows an example of regions that can define an aperture,according to one exemplary embodiment of the present disclosure;

FIG. 37 illustrates post-processing that can be performed after imageshave been captured from an aerial survey, according to one exemplaryembodiment of the present disclosure;

FIG. 38 a shows top and oblique views of sets of scan patterns withsampled sensor pixels, according to one exemplary embodiment of thepresent disclosure;

FIG. 38 b shows top and oblique views of another set of scan patternswith sampled sensor pixels, according to one exemplary embodiment of thepresent disclosure;

FIG. 39 a shows top and oblique views of sets of scan patterns withsensor pixels sampled with a greater number of scan angles than in FIG.38 a , according to one exemplary embodiment of the present disclosure;

FIG. 39 b shows another top and oblique views of sets of scan patternswith sensor pixels sampled with a greater number of scan angles than inFIG. 38 b , according to one exemplary embodiment of the presentdisclosure;

FIG. 40 shows various suitable survey parameters for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 41 shows various suitable survey parameters for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 42 a shows a top down view of a scan pattern, according to oneexemplary embodiment of the present disclosure;

FIG. 42 b shows an oblique view of the scan pattern from FIG. 42 a ,according to one exemplary embodiment of the present disclosure;

FIG. 42 c shows a top down view of a scan pattern, according to oneexemplary embodiment of the present disclosure;

FIG. 42 d shows an oblique view of the scan pattern from FIG. 42 c ,according to one exemplary embodiment of the present disclosure;

FIG. 42 e shows a top down view of a scan pattern, according to oneexemplary embodiment of the present disclosure;

FIG. 42 f shows an oblique view of the scan pattern from FIG. 42 eaccording to one exemplary embodiment of the present disclosure;

FIG. 43 a shows potential scanning mirror structure geometries for asensor having a portrait orientation, according to one exemplaryembodiment of the present disclosure;

FIG. 43 b shows potential scanning mirror structure geometries for asensor having a portrait orientation including one for over-rotation,according to one exemplary embodiment of the present disclosure;

FIG. 43 c shows potential primary mirror geometries for a sensor havinga portrait orientation, according to one exemplary embodiment of thepresent disclosure;

FIG. 43 d shows potential secondary mirror geometries for a sensorhaving a portrait orientation, according to one exemplary embodiment ofthe present disclosure;

FIG. 44 a shows a top down view of scan patterns obtained using ascanning camera system with sensors having a portrait orientation,according to one exemplary embodiment of the present disclosure;

FIG. 44 b shows an oblique view of scan patterns obtained using ascanning camera system with sensors having a portrait orientation,according to one exemplary embodiment of the present disclosure;

FIG. 44 c shows a top down view of multiple scan patterns realisticforward motion, according to one exemplary embodiment of the presentdisclosure;

FIG. 44 d shows an oblique view of multiple scan patterns with realisticforward motion, according to one exemplary embodiment of the presentdisclosure;

FIG. 45 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 45 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 45 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 45 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 45 e shows potential primary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 45 f shows potential secondary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 46 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 46 b shows an oblique view of a scan pattern for the scan driveunit from FIG. 46 a , according to one exemplary embodiment of thepresent disclosure;

FIG. 46 c shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 46 d shows an oblique view of the scan pattern for the scan driveunit from FIG. 46 c , according to one exemplary embodiment of thepresent disclosure;

FIG. 46 e shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 46 f shows an oblique view of the scan pattern for the scan driveunit from FIG. 46 e , according to one exemplary embodiment of thepresent disclosure;

FIG. 47 a shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 47 b shows an oblique view of the scan pattern from FIG. 47 a ,according to one exemplary embodiment of the present disclosure;

FIG. 47 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 47 d shows an oblique view of the scan patterns from FIG. 47 c ,according to one exemplary embodiment of the present disclosure;

FIG. 48 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 48 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 48 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 48 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 48 e shows potential primary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 48 f shows potential secondary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 49 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 49 b shows an oblique view of a scan pattern for the scan driveunit from FIG. 49 a , according to one exemplary embodiment of thepresent disclosure;

FIG. 49 c shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 49 d shows an oblique view of the scan pattern for the scan driveunit from FIG. 49 c , according to one exemplary embodiment of thepresent disclosure;

FIG. 49 e shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 49 f shows an oblique view of the scan pattern for the scan driveunit from FIG. 49 e , according to one exemplary embodiment of thepresent disclosure;

FIG. 50 a shows a scanning camera system from a first perspective,according to one exemplary embodiment of the present disclosure;

FIG. 50 b shows the scanning camera system from a second perspective,according to one exemplary embodiment of the present disclosure;

FIG. 50 c shows the scanning camera system from a third perspective,according to one exemplary embodiment of the present disclosure;

FIG. 50 d shows the scanning camera system from a fourth perspective,according to one exemplary embodiment of the present disclosure;

FIG. 50 e shows a top down view of scan patterns for the scanning camerasystem of FIGS. 50 a-50 d , according to one exemplary embodiment of thepresent disclosure;

FIG. 50 f shows an oblique view of scan patterns for the scanning camerasystem of FIGS. 50 a-50 d , according to one exemplary embodiment of thepresent disclosure;

FIG. 51 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 51 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 51 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 51 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 51 e shows potential primary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 51 f shows potential secondary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 52 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 52 b shows an oblique view of a scan pattern for the scan driveunit from FIG. 52 a , according to one exemplary embodiment of thepresent disclosure;

FIG. 52 c shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 52 d shows an oblique view of the scan pattern for the scan driveunit from FIG. 52 c , according to one exemplary embodiment of thepresent disclosure;

FIG. 52 e shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 52 f shows an oblique view of the scan pattern for the scan driveunit from FIG. 52 e , according to one exemplary embodiment of thepresent disclosure;

FIG. 53 a shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 53 b shows an oblique view of the scan patterns from FIG. 53 a ,according to one exemplary embodiment of the present disclosure;

FIG. 53 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 53 d shows an oblique view of the scan patterns from FIG. 53 c ,according to one exemplary embodiment of the present disclosure;

FIG. 53 e shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 53 f shows an oblique view of the scan patterns from FIG. 53 e ,according to one exemplary embodiment of the present disclosure;

FIG. 54 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 54 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 54 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 54 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 54 e shows potential primary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 54 f shows potential secondary mirror geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 55 a shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 55 b shows an oblique view of the scan patterns from FIG. 55 a ,according to one exemplary embodiment of the present disclosure;

FIG. 55 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 55 d shows an oblique view of the scan patterns from FIG. 55 c ,according to one exemplary embodiment of the present disclosure;

FIG. 55 e shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 55 f shows an oblique view of the scan patterns from FIG. 55 e ,according to one exemplary embodiment of the present disclosure;

FIG. 56 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 56 b shows an oblique view of the scan pattern from FIG. 56 a ,according to one exemplary embodiment of the present disclosure;

FIG. 56 c shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 56 d shows an oblique view of the scan pattern from FIG. 56 c ,according to one exemplary embodiment of the present disclosure;

FIG. 56 e shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 56 f shows an oblique view of the scan pattern from FIG. 56 e ,according to one exemplary embodiment of the present disclosure;

FIG. 57 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 57 b shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 57 c shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 57 d shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 57 e shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 57 f shows an oblique view of the scan patterns from FIG. 57 e ,according to one exemplary embodiment of the present disclosure;

FIG. 58 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 58 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 58 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 58 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 58 e shows scanning mirror structure geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 58 f shows scanning mirror structure geometries including one forover-rotation, according to one exemplary embodiment of the presentdisclosure;

FIG. 59 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 59 b shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 59 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 59 d shows an oblique view of the scan patterns for the scanningcamera system, according to one exemplary embodiment of the presentdisclosure;

FIG. 60 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 60 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 60 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 60 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 60 e shows scanning mirror structure geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 60 f shows scanning mirror structure geometries including one forover-rotation, according to one exemplary embodiment of the presentdisclosure;

FIG. 61 a shows a scan drive unit at a first perspective, according toone exemplary embodiment of the present disclosure;

FIG. 61 b shows the scan drive unit at a second perspective, accordingto one exemplary embodiment of the present disclosure;

FIG. 61 c shows a top down view of a scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 61 d shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 61 e shows scanning mirror structure geometries, according to oneexemplary embodiment of the present disclosure;

FIG. 61 f shows scanning mirror structure geometries including one forover-rotation, according to one exemplary embodiment of the presentdisclosure;

FIG. 62 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 62 b shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 62 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 62 d shows an oblique view of the scan patterns for the scanningcamera system from FIG. 62 c , according to one exemplary embodiment ofthe present disclosure;

FIG. 62 e shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 62 f shows an oblique view of the scan patterns for the scanningcamera system form FIG. 62 e , according to one exemplary embodiment ofthe present disclosure;

FIG. 63 a shows a top down view of a scan pattern for a scan drive unit,according to one exemplary embodiment of the present disclosure;

FIG. 63 b shows an oblique view of the scan pattern for the scan driveunit, according to one exemplary embodiment of the present disclosure;

FIG. 63 c shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;

FIG. 63 d shows an oblique view of the scan patterns for the scanningcamera system from FIG. 63 c , according to one exemplary embodiment ofthe present disclosure;

FIG. 63 e shows a top down view of scan patterns for a scanning camerasystem, according to one exemplary embodiment of the present disclosure;and

FIG. 63 f shows an oblique view of the scan patterns for the scanningcamera system form FIG. 63 e , according to one exemplary embodiment ofthe present disclosure.

DETAILED DESCRIPTION

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). Reference throughoutthis document to “one embodiment”, “certain embodiments”, “anembodiment”, “an implementation”, “an example” or similar terms meansthat a particular feature, structure, or characteristic described inconnection with the embodiment is included in at least one embodiment ofthe present disclosure. Thus, the appearances of such phrases or invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments without limitation.

A scanning camera system may include multiple cameras and coupled beamsteering mechanisms mounted in or on a vehicle. For example, a scanningcamera system may be mounted within a survey hole of an aerial vehicleor in an external space such as a pod. For the sake of clarity, anaerial vehicle will be used to facilitate discussion of the variousembodiments presented herein, though it can be appreciated by one ofskill in the art that the vehicle is not limited to being an aerialvehicle.

A scanning camera system is controlled to capture a series of images ofan object area (typically the ground) as the aerial vehicle follows apath over a survey region. Each image captures a projected region on theobject area with an elevation angle (the angle of the central ray of theimage or ‘line of sight’ to the horizontal plane) and an azimuthal angle(the angle of the central ray around the vertical axis relative to adefined zero azimuth axis). The elevation may also be expressed in termsof the obliqueness (the angle of the central ray of the image or ‘lineof sight’ to the vertical axis), so that vertical imagery with a highelevation corresponds to a low obliqueness and an elevation of 90°corresponds to an obliqueness of 0°. This disclosure will use the groundas the exemplary object area for various embodiments discussed herein,but it can be appreciated that the object does not have to be a groundin other embodiments. For example it may consist of parts of buildings,bridges, walls, other infrastructure, vegetation, natural features suchas cliffs, bodies of water, or any other object imaged by the scanningcamera system.

The calculation of the projected geometry on the object area from acamera may be performed based on the focal length of the lens, the sizeof the camera sensor, the location and orientation of the camera,distance to the object area and the geometry of the object area. Thecalculation may be refined based on nonlinear distortions in the imagingsystem such as barrel distortions, atmospheric effects and othercorrections. Furthermore, if the scanning camera system includes beamsteering elements such as mirrors then these must be taken into accountin the calculation, for example by modelling a virtual camera based onthe beam steering elements to use in place of the actual camera in theprojected geometry calculation.

A scanning camera system may consist of one or more scan drive units,each of which includes a scanning element such as a scanning mirror toperform beam steering. A scanning mirror may be driven by any suitablerotating motor (such as a piezo rotation stage, a stepper motor, DCmotor or brushless motor) coupled by a gearbox, direct coupled or beltdriven. Alternatively the mirror may be coupled to a linear actuator orlinear motor via a gear. Each scan drive unit includes a lens to focuslight beams onto one or more camera sensors, where the lens may beselected from the group comprising: a dioptric lens, a catoptric lensand a catadioptric lens. Each scan drive unit also includes one or morecameras that are configured to capture a series of images, or frames, ofthe object area. Each frame has a view elevation and azimuth determinedby the scan drive unit geometry and scan angle, and may be representedon the object area by a projected geometry. The projected geometry isthe region on the object area imaged by the camera.

The projected geometry of a sequence of frames captured by a scan driveunit may be combined to give a scan pattern. Referring now to thedrawings, where like reference numerals designate identical orcorresponding parts throughout the several views, FIG. 1 a shows thescan patterns for a scanning camera system 300 with three scan driveunits 301, 302, 303 from a top down view (left) and a perspective view(right) showing an aerial vehicle 110. It is noted that the scanpatterns in FIG. 1 a assume all frames are captured for the same aerialvehicle 110 location. In a real system, the aerial vehicle 110 will movebetween frame captures as will be discussed later. The x- and y-axes inthe plot meet at the location on the ground directly under the aerialvehicle 110. The grid lines 117, 118 correspond to a distance to theleft and right of the aerial vehicle 110 equal to the altitude of theaerial vehicle 110. Similarly, the grid lines 119, 116 correspond to adistance forward and behind the aerial vehicle 110 equal to the altitudeof the aerial vehicle 110. The two curved scan patterns 111, 112correspond to the two cameras of the scan drive unit 301, while the twoscan patterns 113, 114 are symmetric about the y-axis and correspond tothe single camera of each of scan drive unit 302 and scan drive unit303. The dashed single projective geometry 115 corresponds to a lowerresolution overview camera image.

The aerial vehicle 110 may follow a serpentine flight path such as theone illustrated in FIG. 2 . The path consists of a sequence of straightflight lines 210, 211, 212, 213, 214, 215 along a flight direction (they-axis) connected by curved turning paths 220, 221, 222, 223, 224, 225.The serpentine flight path is characterised by a flight line spacing226, that is the spacing of adjacent flight lines (210 to 211, 211 to212, etc.) perpendicular to the flight direction (i.e. along the x-axisin FIG. 2 ). In general, the flight line spacing is fixed, but may beadaptive to capture some regions with an increased density of images. Itis noted that the combined width of the scan patterns may be much widerthat the flight line spacing.

Each scan pattern is repeated as the aerial vehicle moves along itsflight path over the survey area to give a dense coverage of the scenein the survey area with a suitable overlap of captured images forphotogrammetry, forming photomosaics and other uses. Across the flightline this can be achieved by setting the scan angles of frames within ascan pattern close enough together. Along the flight lines this can beachieved by setting a forward spacing between scan patterns (i.e. setsof frames captured as the scan angle is varied) that is sufficientlysmall. The timing constraints of each scan drive unit may be estimatedbased on the number of frames per scan pattern, the forward spacing andthe speed of the aerial vehicle over the ground. The constraints mayinclude a time budget per frame capture and a time budget per scanpattern.

FIG. 1 b shows the scan patterns of the scanning camera system 300 fromFIG. 1 a with additional scan patterns for each scan drive unit 301,302, 303 positioned one forward spacing ahead and behind the originalobject area geometry. In this configuration the scan angle steps andforward spacings are selected to give a 10% overlap of frames. In otherconfigurations, the scan angle steps and forward spacings may beselected to give a fixed number of pixels of overlap in frames, or anoverlap corresponding to a specified distance on the object area, orsome other criteria.

In general, the timing constraints of scanning camera systems have morerestrictive timing constraints than fixed camera systems. However,scanning camera systems may allow an increased flight line spacing for agiven number of cameras resulting in a more efficient camera system.They also make more efficient use of the limited space in which they maybe mounted in a commercially available aerial vehicle (eitherinternally, such as in a survey hole, or externally, such as in a pod).

The flight lines 210, 211, 212, 213, 214, 215 of the serpentine flightpath shown in FIG. 2 are marked with locations spaced at the appropriateforward spacings for the three scan drive units 301, 302, 303. These maybe considered to mark the position of the aerial vehicle 110 on theserpentine flight path at which the initial frame of each scan patternwould be captured for each of the three scan drive units 301, 302, 303.The forward spacing used for the scan drive units 302, 303 thatcorrespond to scan patterns 113, 114 in FIG. 1 a is approximately halfof the forward spacing used for the scan drive unit 301 corresponding tothe two curved scan patterns 111, 112 of FIG. 1 a for an equalpercentage of forward overlap of scan angles.

The flight lines of the serpentine path may take any azimuthalorientation. It may be preferable to align the flight lines (y-axis inFIG. 1 a and FIG. 1 b ) with either a North Easterly or North Westerlydirection. In this configuration the scanning camera system 300illustrated in FIG. 1 a and FIG. 1 b has advantageous properties for thecapture of oblique imagery aligned with the cardinal directions (North,South, East and West).

FIG. 3 shows the distribution of views (elevation and azimuth) at ninedifferent ground locations for a scanning camera system 300 with scanpatterns as shown in FIG. 1 a , and flown with a more realisticserpentine flight path (more and longer flight lines) than the examplesurvey flight path of FIG. 2 . Each plot is a Lambert equal areaprojection with y-axis parallel to the flight lines. The point atcoordinate x=0, y=0 corresponds to a view of the ground directly beneaththe aerial vehicle 110 with zero obliqueness.

The circles of viewing directions at fixed elevations 236, 237, 238represent views with obliqueness of 12°, 39° and 51°, respectively. Thecurved path of viewing directions in the hemisphere 294, 295, 296, 297represent views with obliqueness between 39° and 51° spaced at 90°azimuthally. The curved path of viewing directions in the hemisphere294, 295, 296, 297 may represent suitable views for oblique imageryalong cardinal directions if the serpentine flight follows a NorthEasterly or North Westerly flight line direction.

Each viewing direction 230, 231, 232, 233, 234, 235 corresponds to apixel in an image captured by the scanning camera system 300 andrepresents the view direction (elevation and azimuth) of that groundlocation at the time of image capture relative to the aerial vehicle 110in which the scanning camera system 300 is mounted. Neighbouring pixelsin the image would correspond to neighbouring ground locations withsimilar view directions. The viewing directions 230, 231, 232, 233, 234,235 either fall within a horizontal band through the centre or acircular band around 45-degree elevation. Viewing directions 230, 235 inthe horizontal band correspond to images captured by the cameras of scandrive unit 302 and scan drive unit 303, while viewing directions 231,232, 233, 234 around the circular band correspond to images captured byscan drive unit 301. Some views may be suitable for oblique imagery(e.g. viewing direction 231, 232, 233, 234) and some for verticalimagery (e.g. viewing direction 235). Other views may be suitable forother image products, for example they may be useful in the generationof a 3D textured model of the area.

The capture efficiency of aerial imaging is typically characterized bythe area captured per unit time (e.g. square km per hour). For aserpentine flight path with long flight lines, a good rule of thumb isthat this is proportional to the speed of the aircraft and the flightline spacing, or swathe width of the survey. A more accurate estimatewould account for the time spent manoeuvring between flight lines.Flying at increased altitude can increase the efficiency as the flightline spacing is proportional to the altitude and the speed can alsoincrease with altitude, however it would also reduce the resolution ofthe imagery unless the optical elements are modified to compensate (e.g.by increasing the focal length or decreasing the sensor pixel pitch).

The data efficiency of a scanning camera system may be characterised bythe amount of data captured during a survey per area (e.g. gigabyte (GB)per square kilometre (km)). The data efficiency increases as the overlapof images decreases and as the number of views of each point on theground decreases. The data efficiency determines the amount of datastorage required in a scanning camera system for a given survey, andwill also have an impact on data processing costs. Data efficiency isgenerally a less important factor in the economic assessment of runninga survey than the capture efficiency as the cost of data storage andprocessing is generally lower than the cost of deploying an aerialvehicle with a scanning camera system.

The maximum flight line spacing of a given scanning camera system may bedetermined by analysing the combined projection geometries of thecaptured images on the ground (scan patterns) along with the elevationand azimuth of those captures, and any overlap requirements of theimages such as requirements for photogrammetry methods used to generateimage products.

In order to generate high quality imaging products, it may be desirableto: (1) image every point on the ground with a diversity of captureelevation and azimuth, and (2) ensure some required level of overlap ofimages on the object area (e.g. for the purpose of photogrammetry orphotomosaic formation)

The quality of an image set captured by a given scanning camera systemoperating with a defined flight line spacing may depend on variousfactors including image resolution and image sharpness.

The image resolution, or level of detail captured by each camera, istypically characterized by the ground sampling distance (GSD), i.e. thedistance between adjacent pixel centres when projected onto the objectarea (ground) within the camera's field of view. The calculation of theGSD for a given camera system is well understood and it may bedetermined in terms of the focal length of the camera lens, the distanceto the object area along the line of sight, and the pixel pitch of theimage sensor. The distance to the object area is a function of thealtitude of the aerial camera relative to the ground and the obliquenessof the line of sight.

The sharpness of the image is determined by several factors including:the lens/sensor modular transfer function (MTF); the focus of the imageon the sensor plane; the surface quality (e.g. surface irregularitiesand flatness) of any reflective surfaces (mirrors); the stability of thecamera system optical elements; the performance of any stabilisation ofthe camera system or its components; the motion of the camera systemrelative to the ground; and the performance of any motion compensationunits.

The combined effect of various dynamic influences on an image capturemay be determined by tracking the shift of the image on the sensorduring the exposure time. This combined motion generates a blur in theimage that reduces sharpness. The blur may be expressed in terms of adrop in MTF. Two important contributions to the shift of the image arethe linear motion of the scanning camera system relative to the objectarea (sometimes referred to as forward motion) and the rate of rotationof the scanning camera system (i.e. the roll, pitch and yaw rates). Therotation rates of the scanning camera system may not be the same as therotation rates of the aerial vehicle if the scanning camera system ismounted on a stabilisation system or gimbal.

The images captured by a scanning camera system may be used to create anumber of useful image based products including: photomosaics includingorthomosaic and panoramas; oblique imagery; 3D models (with or withouttexture); and raw image viewing tools.

In addition to the resolution and sharpness, the quality of the capturedimages for use to generate these products may depend on other factorsincluding: the overlap of projected images; the distribution of views(elevations and azimuths) over ground points captured by the camerasystem during the survey; and differences in appearance of the area dueto time and view differences at image capture (moving objects, changedlighting conditions, changed atmospheric conditions, etc.).

The overlap of projected images is a critical parameter when generatingphotomosaics. It is known that the use of a low-resolution overviewcamera may increase the efficiency of a system by reducing the requiredoverlap between high resolution images required for accuratephotogrammetry. This in turn improves the data efficiency and increasesthe time budgets for image capture.

The quality of the image set for vertical imagery depends on thestatistics of the obliqueness of capture images over ground points. Anydeviation from the zero obliqueness results in vertical walls ofbuildings being imaged, resulting in a leaning appearance of thebuildings in the vertical images. The maximum obliqueness is the maximumdeviation from vertical in an image, and is a key metric of the qualityof the vertical imagery. The maximum obliqueness may vary between 10°for a higher quality survey up to 25° for a lower quality survey. Themaximum obliqueness is a function of the flight line spacing and theobject area projective geometry of captured images (or the scanpatterns) of scan drive units.

An orthomosaic blends image pixels from captured images in such a way asto minimise the obliqueness of pixels used while also minimisingartefacts where pixel values from different original capture images areadjacent. The maximum obliqueness parameter discussed above is thereforea key parameter for orthomosaic generation, with larger maximumobliqueness resulting in a leaning appearance of the buildings. Thequality of an orthomosaic also depends on the overlap of adjacent imagescaptured in the survey. A larger overlap allows the seam between pixelstaken from adjacent images to be placed judiciously where there islittle texture, or where the 3D geometry of the image is suitable forblending the imagery with minimal visual artefact. Furthermore,differences in appearance of the area between composited image pixelsresult in increased artefacts at the seams also impacting the quality ofthe generated orthomosaic.

The quality of imagery for oblique image products can be understoodalong similar lines to that of vertical imagery and orthomosaics. Someoblique imagery products are based on a particular viewpoint, such as a45-degree elevation image with azimuth aligned with a specific direction(e.g. the four cardinal directions North, South, East or West). Thecaptured imagery may differ from the desired viewpoint both in elevationand azimuth. Depending on the image product, the loss of quality due toerrors in elevation or azimuth will differ. Blended or stitched imageoblique products (sometimes referred to as panoramas) may also begenerated. The quality of the imagery for such products will depend onthe angular errors in views and also on the overlap between image viewsin a similar manner to the discussion of orthomosaic imagery above.

The quality of a set of images for the generation of a 3D model isprimarily dependent on the distribution of views (elevation and azimuth)over ground points. In general, it has been observed that decreasing thespacing between views and increasing the number of views will bothimprove the expected quality of the 3D model. Heuristics of expected 3Dquality may be generated based on such observations and used to guidethe design of a scanning camera system.

FIGS. 4 a-4 f, 5 a-5 f and 6 a-6 f demonstrate the scan drive units 301,302, 303 that can be used to achieve the scan patterns of FIG. 1 a . Thefirst scan drive unit 301, shown in FIGS. 4 a and 4 b , can be used tocapture scan patterns 111, 112 having circular arcs centred around anelevation of 45°. Top down and oblique views of the scan patterns 111,112 from the two cameras 310, 311 of scan drive unit 301 are shown inFIGS. 4 c and 4 d , respectively.

Two geometric illustrations of the scan drive unit 301 from differentperspectives are shown in FIGS. 4 a and 4 b . The scan drive unit 301comprises a scanning mirror structure 312 attached to a scan drive 313on a vertical scan axis (elevation θ_(S)=−90° and azimuth ϕ_(S)=0°). Inone embodiment, the scanning mirror structure 312 is double-sided. Thegeometric illustration shows the configuration with the scan angle ofthe scan drive 313 set to 0° so that the first mirror surface 314 isoriented (elevation θ_(M) ¹=0° and azimuth ϕ_(M) ¹=0°) with its normaldirected toward the first camera 310 along the y-axis. A second mirrorsurface 315 is mounted on the opposite side of the scanning mirrorstructure 312 and directed toward the second camera 311. The two cameras310, 311 are oriented downward at an oblique angle but with opposingazimuths (camera 310 elevation θ_(S)=−45° and azimuth ϕ_(S)=180°, camera311 elevation θ_(S)=−45° and azimuth ϕ_(S)=0°).

In one example, the cameras 310, 311 utilise the Gpixel GMAX3265 sensor(9344 by 7000 pixels of pixel pitch 3.2 microns). The camera lenses mayhave a focal length of 420 mm and aperture of 120 mm (corresponding toF3.5). The scanning mirror structure 312 may have a thickness of 25 mm.Unless otherwise stated, all illustrated cameras utilise the GpixelGMAX3265 sensor, with a lens of focal length 420 mm and aperture of 120mm (F3.5), and all mirrors illustrated have a thickness of 25 mm.

The optical axis of a lens is generally defined as an axis of symmetryof the lens. For example it may be defined by a ray passing from a pointat or near the centre of the sensor through the lens elements at or nearto their centres. The optical axis of a lens in a scan drive unit may bemodified by one or more mirror structures of the scan drive unit. It mayextend beyond the lens, reflect at one or more mirror surfaces, thencontinue to a point on the object area. The distance from the camera 310to the mirror surface 314 along the optical axis may be 247 mm. Thedistance from the second camera 311 to the second mirror surface 315along the optical axis may also 247 mm. In other embodiments, thedistances between elements may be selected in order that the componentsfit within the required space, and the scan drive unit 301 is able torotate by the required angular range (which may be between ±30.7° and±46.2° for the two sided arrangement described here). The scanningmirror structure 312 rotation axis is assumed to intersect the opticalaxis of one or both cameras 310, 311. The distances between componentsof all scan drive units presented in this specification may be selectedto best fit within the available space while allowing the requiredangular range of rotation of the scanning mirror structure.

The shape of the reflective surface of the scanning mirror structureshould be large enough to reflect the full beam of rays imaged from thearea on the ground onto the camera lens aperture so they are focusedonto the camera sensor as the scan angle of the scan drive unit variesover a given range of scan angles. In one embodiment of scanning mirrorstructure 312, the standard range of scan angles is −30.7° to 30.7°.Existing methods have been described elsewhere that may be used tocalculate a suitable scanning mirror structure shape for which thiscriterion is met.

One suitable method determines the geometry of regions of the scanningmirror structure surface that intersects the beam profile defined byrays passing between the object area and the camera sensor through thelens aperture at each sampled scan angle. The beam profile may vary fromcircular at the aperture of the camera, to a rectangular shapecorresponding to the sensor shape at the focus distance. The union ofthe geometries of these intersection regions on the mirror surface givesthe required scanning mirror structure size to handle the sampled set ofscan angles. In some instances, the calculated scanning mirror structureshape may be asymmetric about the axis of rotation, and so it may bepossible to reduce the moment of inertia of the scanning mirrorstructure by shifting the axis of rotation. In this case, the scanningmirror structure geometry may be re-calculated for the shifted axis ofrotation. The re-calculated shape may still be asymmetric around theaxis of rotation, in which case the process of shifting the axis ofrotation and re-calculating the geometry may be iterated until thescanning mirror structure is sufficiently close to symmetric and themoment of inertia is minimised.

The methods described above generate the geometry of the scanning mirrorstructure required for a particular sensor orientation in the camera.The sensors of the scan drive units 301, 302, 303 shown in FIGS. 4 a-4f, 5 a-5 f and 6 a-6 f are oriented in what may be referred to as alandscape orientation. Viewed from above, the projected geometry of theimage captured closest to the y-axis has a landscape geometry (it iswider along the x-axis than it is long along the y-axis). Alternativeembodiments may use a sensor oriented at 90° to that illustrated inFIGS. 4 a-4 f, 5 a-5 f and 6 a-6 f , referred to as a portraitorientation. Viewed from above, the projected geometry of the imagecaptured closest to the y-axis would have a portrait geometry (it isnarrower along the x-axis than it is long along the y-axis). Otherembodiments may use any orientation between landscape and portraitorientation.

It may be advantageous to use a scanning mirror structure geometry thatis large enough to handle the portrait orientation of the sensor inaddition to the landscape orientation. Such a scanning mirror structuregeometry may be generated as the union of the landscape orientation andportrait orientation mirror geometries. Such a scanning mirror structuregeometry may allow greater flexibility in the configuration of the scandrive use. Further, it may be advantageous to use a scanning mirrorstructure geometry that can handle any orientation of the sensor byconsidering angles other than the landscape and portrait orientations.Such a scanning mirror structure can be calculated assuming a sensorthat is circular in shape with a diameter equal in size to the diagonallength of the sensor.

The scanning mirror structure may comprise aluminium, beryllium, siliconcarbide, fused quartz or other materials. The scanning mirror structuremay include hollow cavities to reduce mass and moment of inertia, or besolid (no hollow cavities) depending on the material of the scanningmirror structure. The mirror surface may be coated to improve thereflectivity and or flatness, for example using nickel, fused quartz orother materials. The coating may be on both sides of the scanning mirrorstructure to reduce the thermal effects as the temperature of thescanning mirror structure changes. The required flatness of the mirrorsurface may be set according to the required sharpness of the captureimages and the acceptable loss of sharpness due to the mirrorreflection. The mirror surface may be polished to achieve the requiredflatness specification.

The thickness of a scanning mirror structure is generally set to be assmall as possible, so as to reduce mass and minimise spatialrequirements, while maintaining the structural integrity of the scanningmirror structure so that it can be dynamically rotated within the timebudget of the captured images of the scan patterns without compromisingthe optical quality of captured images. In one embodiment, a thicknessof 25 mm may be suitable.

Depending on the manufacturing process and materials used in thefabrication of the scanning mirror structure, it may be advantageous touse a convex mirror shape. In this case, the convex hull of the shapecalculated above may be used as the scanning mirror structure shape.Furthermore, the scanning mirror structure shape may be dilated in orderto ensure that manufacturing tolerances in the scanning mirror structureand other components of the scan drive unit or control tolerances insetting the scan angle do not result in any stray or scattered rays inthe system and a consequent loss of visual quality.

FIG. 4 e shows various scanning mirror structure geometries calculatedfor the scan drive unit 301. These include the minimum geometry (“min”),a dilated minimum geometry that is extended by 5 mm beyond the minimumgeometry around its perimeter (“dilate”) and a dilated convex geometrythat is the convex hull of the dilated minimum geometry (“convex”). Anyof these geometries, or other variants that may be envisaged (e.g. tohandle alternative sensor orientations), may be used to define the shapeof the scanning mirror structure 312 for this scan drive unit 301.

The axis of rotation 316 was selected such that it intersects the rayalong the optical axis of the lens through the centre of the aperture.The scan drive unit would be attached at the end that extends beyond thescanning mirror structure 312. The centre of mass of the scanning mirrorstructure 312 is aligned with the axis of rotation 316, so that no shiftof the axis of rotation is required.

FIG. 4 f shows the dilated convex geometry again (“convex”), and also anextended geometry that might be required if the range of scan angles isextended by 7.5° at each end of the scan angle range (“over”). Theangular spacing of the scan angle samples is kept roughly the same asthe original in the calculation by increasing the number of samplesteps. This geometry will be discussed further later in thisspecification with reference to over-rotation for yaw correction.

FIG. 4 g shows a magnified view of additional geometries of mirrorsand/or paddle flaps, according to an embodiment. For example, as can beseen in FIG. 4 g , paddle flaps (hatched line areas) can cover an entireperimeter of a mirror, or one or more portions thereof. The mirrorsand/or paddle flaps can be symmetric or asymmetric.

The capture of images on opposite mirror surfaces (e.g. mirror surface314, 315) may be synchronised or not synchronised. In general the imagecapture takes place once the scanning mirror structure has comecompletely to rest in order to achieve a high image quality. In otherarrangements, image stabilisation may be used to compensate for mirrormotion during image exposure.

In a slightly modified arrangement, the scanning mirror structure 312may employ a single mirror surface (i.e. one of mirror surface 314 or315) and the scanning mirror structure 312 may rotate through a full360°, using the scan drive 313, so that the single mirror surface may beused in turn by the two cameras 311, 310. For example, in a modifiedarrangement, the second mirror surface 315 does not need to be a mirrorsurface. This multiplexing arrangement would have tighter requirementson the timing of image capture as the images are not capturedsimultaneously for both mirror surfaces 314, 315.

The second scan drive unit 302 of the scanning camera system 300 isshown in FIG. 5 a-5 f . As shown in FIGS. 5 c and 5 d , scan drive unit302 can be used to capture a single straight scan pattern 113 at a rightangle to the flight line from 0 to 45° obliqueness. The scan pattern 113extends to the right of the aerial vehicle 110 looking ahead along theflight line. Two geometric illustration of the scan drive unit 302 fromdifferent perspectives are shown in FIG. 5 a and FIG. 5 b . The scandrive unit 302 comprises a single sided scanning primary mirror 323 heldon a horizontal scan axis (elevation θ_(S)=−0° and azimuth ϕ_(S)=180°),and a fixed secondary mirror 324. The geometric illustration shows theconfiguration with the scan angle of the scan drive 322 set to 0° atwhich angle the primary mirror's 323 surface is oriented with a normaldirected at an oblique between the z- and x-axes (elevation θ_(M) ¹=−45°and azimuth ϕ_(M) ¹=90°). The secondary mirror 324 is oriented with anormal opposing that of the primary mirror 323 when the scan angle is 0°(elevation θ_(M) ¹=45° and azimuth ϕ_(M) ¹=−90°). There is a singlecamera 321 which is directed downwards at an angle of 1 degree to thevertical z-axis (elevation θ_(S)=−89° and azimuth ϕ_(S)=)—90°. Scandrive 322 samples scan angles from −23° to −0.5° in order to generatethe scan pattern 113.

In one embodiment, the distance from the lens of camera 321 to thesecondary mirror 324 along the optical axis may be 116 mm, and thedistance from the primary mirror 323 to secondary mirror 324 may be 288mm along the optical axis. Of course, other distances may be used inother embodiments.

There are two mirror geometries to consider for scan drive unit 302.Example geometries of the (scanning) primary mirror 323 are shown inFIG. 5 e , including the minimal geometry (“min”), dilated geometry(“dilate”) and convex geometry (“convex”), which is essentially the sameas the dilated geometry. The centroid of the computed primary mirror wasfound to be shifted relative to the scan drive axis projected to themirror surface, so FIG. 5 e shows a shifted scan drive axis that may beused to reduce the moment of inertia as discussed above. Examplegeometries of the (fixed) secondary mirror 324 are shown in FIG. 5 f ,including the minimum geometry (“min”) and dilated geometry (“dilate”).

The third scan drive unit 303, illustrated in FIGS. 6 a and 6 b , is aclone of the second scan drive unit 302 rotated by 180° around thez-axis. FIGS. 6 a and 6 b include camera 325, primary mirror 327, scandrive 326, and secondary mirror 328. As shown in FIGS. 6 c and 6 d , dueto the symmetry of the scan drive units 302, 303, the scan pattern 114for scan drive unit 303 is a mirror image of scan pattern 113 for scandrive unit 302, following a straight path that extends to the left ofthe aerial vehicle 110 looking forward along the flight line. The mirrorgeometries and dynamics shown in FIGS. 6 e and 6 f are identical tothose described with reference to FIGS. 5 e and 5 f above.

FIGS. 7 a to 7 d show a range of perspective views of the combinedcomponents of scan drives 301, 302, 303 of the scanning camera system300 that were described with respect to FIGS. 4 a-4 f, 5 a-5 f, and 6a-6 f above including: cameras 310, 311, 321, 325; scanning mirrorstructure 312 with mirror surfaces 314, 315 attached to a scan drive313; two primary mirrors 323, 327 attached to scan drives 322, 326; andtwo fixed secondary mirrors 324, 328.

It can be seen in FIGS. 7 a-7 d that the scan drive unit 302 structureis arranged so that it's imaging path passes under camera 310 of scandrive unit 301, and scan drive unit 303 is arranged so that it's imagingpath passes under camera 311 of scan drive unit 301. This arrangement ishighly efficient spatially and advantageous for deployment in a widerange of aerial vehicle camera (survey) holes.

FIGS. 7 e and 7 f show the scan patterns achieved using the scanningcamera system 300 including curved scan patterns 111, 112 of obliqueimagery, and straight scan patterns 113, 114 that capture a sweep ofimages from vertical to oblique along a direction perpendicular to theflight line. Further to the scan drive unit imaging capability, thescanning camera system 300 may additionally include one or more fixedcameras. These cameras may be standard RGB cameras, infrared cameras,greyscale cameras, multispectral cameras, hyperspectral cameras or othersuitable cameras. In one embodiment, fixed camera may be a Phase OneiXM100 camera sensor (11664×8750 pixels of 3.76 micron pitch) with an 80mm F5.6 lens. Single or multipoint LIDAR camera systems may also beincorporated into the scanning camera system.

The fixed camera may be used as an overview camera, and the capture rateof the fixed camera may be set in order to achieve a desired forwardedoverlap between captured images, such as 60%. The flight line spacing ofthe survey may be limited such that the sideways overlap of overviewcamera images achieves a second desired goal, such as 40%. The overviewcamera may be directed vertically downward and may be rotated about thevertical axis such that the projected geometry on the object area is notaligned with the orientation of the aerial vehicle.

The scan patterns 111, 112, 113, 114 of the scanning camera system 300described above with respect to FIGS. 1 a, 4 c, 4 d, 5 c, 5 d, 6 c, 6 d,7 e and 7 f did not represent the forward motion of the aerial vehicle110; they were generated assuming a fixed aerial vehicle 110 above theobject area. Replotting the ground projection geometry of the scanpatterns to include the aerial vehicle 110 linear motion over the groundmay give the slightly modified scan pattern plots of FIG. 8 a (singlescan pattern case) and FIG. 8 b (three scan patterns case). These scanpatterns give a more realistic view of the scan patterns that may beused to compute the flight parameters to achieve an overlap target (suchas 10% overlap). It is noted that they do not affect the view directions(elevation and azimuth) of captured images as the view angle iscalculated as a function of the difference in location of the imagedground points relative to the location of the aerial vehicle 110 at thetime of capture of an image. FIG. 8 c shows top down and oblique viewsof multiple sets of scan patterns captured by a scanning camera systemaccording to one exemplary embodiment of the present disclosure. Thescanning camera system of FIG. 8 c is a reduced system comprising scandrive unit 301 without camera 311 and scan drive unit 302 only. Thisscanning camera system may be flown in a modified flight path where eachflight line 210 to 215 is flown in both directions.

It is understood that the scanning camera system 300 geometry may bemodified in a number of ways without changing the essentialfunctionality of each of the scan drive units 301, 302, 303. Forexample, the scan drive and mirror locations and thicknesses may bealtered, the distances between elements may be changed, and the mirrorgeometries may change. In general it is preferable to keep the mirrorsas close together and as close to the lens as is feasible withoutresulting in mechanical obstructions that prevent the operationallydesired scan angle ranges or optical obstructions that result in loss ofimage quality.

Furthermore, changes may be made to the focal distances of theindividual lenses or the sensor types and geometries. In addition tocorresponding geometric changes to the mirror geometries and locations,these changes may result in changes to the appropriate flight linedistances, steps between scan angles, range of scan angles, and frametiming budgets for the system.

A scanning camera system may be operated during a survey by a systemcontrol 405. A high-level representation of a suitable system control405 is shown in FIG. 9 . Components enclosed in dashed boxes (e.g.auto-pilot 401, motion compensation (MC) unit 415) represent units thatmay be omitted in other embodiments. The system control 405 may haveinterfaces with the scanning camera system 408, stabilisation platform407, data storage 406, GNSS receiver 404, auto-pilot 401, pilot display402 and pilot input 403. The system control 405 may comprise one or morecomputing devices that may be distributed, such as computers, laptopcomputers, micro controllers, ASICS or FPGAs, to control the scan driveunits and fixed cameras of the camera system during operation. Thesystem control 405 can also assist the pilot or auto-pilot of the aerialvehicle to follow a suitable flight path over a ground region ofinterest, such as the serpentine flight path discussed with respect toFIG. 2 . The system control 405 may be centrally localised ordistributed around the components of the scanning camera system 408. Thesystem control 405 may use Ethernet, serial, CoaxPress (CXP), CAN Bus,i²C, SPI, GPIO, custom internal interfaces or other interfaces asappropriate to achieve the required data rates and latencies of thesystem.

The system control 405 may include one or more interfaces to the datastorage 406, which can store data related to survey flight path, scandrive geometry, scan drive unit parameters (e.g. scan angles), DigitalElevation Model (DEM), Global Navigation Satellite System (GNSS)measurements, inertial measurement unit (IMU) measurements,stabilisation platform measurements, other sensor data (e.g. thermal,pressure), motion compensation data, mirror control data, focus data,captured image data and timing/synchronisation data. The data storage406 may also include multiple direct interfaces to individual sensors,control units and components of the scanning camera system 408.

The scanning camera system 408 may comprise one or more scan drive units411, 412, an IMU 409 and fixed camera(s) 410. The IMU 409 may compriseone or more individual units with different performance metrics such asrange, resolution, accuracy, bandwidth, noise and sample rate. Forexample, the IMU 409 may comprise a KVH 1775 IMU that supports a samplerate of up to 5 kHz. The IMU data from the individual units may be usedindividually or fused for use elsewhere in the system. In oneembodiment, the fixed camera(s) 410 may comprise a Phase One iXM100,Phase One iXMRS100M, Phase One iXMRS150M, AMS Cmosis CMV50000, GpixelGMAX3265, or IOIndustries Flare 48M30-CX and may use a suitable cameralens with focal length between 50 mm and 200 mm.

The system control 405 may use data from one or more GNSS receivers 404to monitor the position and speed of the aerial vehicle 110 in realtime. The one or more GNSS receivers 404 may be compatible with avariety of space-based satellite navigation systems, including theGlobal Positioning System (GPS), GLONASS, Galileo and BeiDou.

The scanning camera system 408 may be installed on a stabilisationplatform 407 that may be used to isolate the scanning camera system 408from disturbances that affect the aerial vehicle 110 such as attitude(roll, pitch, and/or yaw) and attitude rate (roll rate, pitch rate, andyaw rate). It may use active and/or passive stabilisation methods toachieve this. Ideally, the scanning camera system 408 is designed to beas well balanced as possible within the stabilisation platform 407. Inone embodiment the stabilisation platform 407 includes a roll ring and apitch ring so that scanning camera system 408 is isolated from roll,pitch, roll rate and pitch rate disturbances.

In some embodiments the system control 405 may further control thecapture and analysis of images for the purpose of setting the correctfocus of lenses of the cameras of the scan drive units 411, 412 and/orfixed camera(s) 410. The system control 405 may set the focus onmultiple cameras based on images from another camera. In otherembodiments, the focus may be controlled through thermal stabilisationof the lenses or may be set based on known lens properties and anestimated optical path from the camera to the ground. Some cameras ofthe scanning camera system 408 may be fixed focus. For example, some ofthe fixed focus cameras used for overview images may be fixed focus.

Each scanning camera system is associated with some number of scan driveunits. For example scanning camera system 408 includes scan drive unit411, 412, though more can be included. As another example, the scanningcamera system 300 shown in FIG. 7 a-7 d comprises 3 scan drive units301, 302, 303 that were discussed above with respect to FIGS. 4 a-4 f, 5a-5 f and 6 a-6 f . Alternative configurations of scanning camerasystems with different numbers of scan drive units will be discussedbelow. Each scan drive unit 411, 412 shown in FIG. 9 may comprise ascanning mirror 413 and one or more cameras 414, 416.

Each camera 414, 416 of FIG. 9 may comprise a lens, a sensor, andoptionally a motion compensation unit 415, 417. The lens and sensor ofthe cameras 414, 416 can be matched so that the field of view of thelens is able to expose the required area of the sensor with someacceptable level of uniformity.

Each lens may incorporate a focus mechanism and sensors to monitor itsenvironment and performance. It may be thermally stabilised and maycomprise a number of high-quality lens elements with anti-reflectivecoating to achieve sharp imaging without ghost images from internalreflections. The system control 405 may perform focus operations basedon focus data 438 between image captures. This may use known techniquesfor auto-focus based on sensor inputs such as images (e.g. imagetexture), LIDAR, Digital Elevation Model (DEM), thermal data or otherinputs.

The control of the scanning mirror 413 and the capture of images by thecamera or cameras 414, 416 of the scan drive unit 411 are illustrated inthe high-level process of FIG. 10 . The system control 405 uses datainputs from data storage 406 to iteratively set the scan angle 430 andtrigger the camera or cameras 414, 416 to capture images. The scan angle430 is set according to the scan drive unit parameters 434, whichdefines the sequence of scan drive angles corresponding to the sequenceof images to be captured for each scan pattern, and the sequentialtiming of frames of the scan pattern. As discussed above, the sequenceof scan angles and timing of frame capture may be set to achieve adesired overlap of projective geometry of captured images on the groundthat is advantageous for particular aerial image products.

Optionally, the sequence of scan angle 430 settings may be updatedaccording to IMU data such as the attitude of the aerial vehiclerelative to the expected attitude (aligned with the flight line). Forexample, the scan angle 430 may be corrected to account for the yaw ofthe aerial vehicle in the case that the stabilisation platform 407 doesnot handle yaw. Specifically, for the scan drive unit 301 discussed inrelation to FIG. 4 a-4 f that captures two arc shaped scan patterns 111,112, a scan angle correction of half of the yaw angle may be used sothat the scan pattern is corrected for yaw as will be discussed ingreater detail later with respect to FIGS. 32-37 . Alternatively, if thestabilisation platform 407 has only partial yaw correction then asmaller scan angle correction may be used.

The mirror control 432 receives an instruction to set the scan drive tothe scan angle 430 from the system control 405, and optionally usesinputs from a mirror sensor 433 that reports the status of mirror drive431 in order to control the mirror drive 431 so that the scanning mirror413 is set to the desired scan angle 430. The mirror control 432 sendsmirror control data 437 to be stored in data storage 406. When thescanning mirror 413 has settled to the correct scan angle according tothe mirror control data 437, the system control 405 may send a triggerinstruction to the camera or cameras 414, 416 associated with thescanning mirror 413.

Optionally, the system control 405 also controls the timing of thecamera trigger to be synchronous with the operation of the motioncompensation of each camera 414, 416. Motion compensation (MC) data 435relating to the motion compensation for the camera 414, 416 is stored indata storage 406 and may be used to achieve this synchronisation.

Pixel data 439 corresponding to captured images are stored in the datastorage 406. Optionally, gimbal angles 470 may be stored in data storage406 including information relating to the orientation of the scanningcamera system 408 in the stabilisation platform 407 (i.e. gimbal) at thetime of capture of images for the stored pixel data 439. Other datalogged synchronously with the image capture may include GNSS data(ground velocity 462, latitude/longitude data 463 and altitude 464 asshown in FIG. 11 ) and IMU attitude data 436.

It may be understood that the process illustrated in FIG. 10 may beemployed to capture motion compensated images with projective geometryaccording to the scan patterns of the scan drive unit. This process maybe slightly modified without affecting the scope of the systems andmethods described in this specification.

The motion compensation may use a variety of methods including, but notlimited to, tilting or rotating transparent optical plates or lenselements in the optical path, tilting or rotating mirrors in the opticalpath, and/or camera sensor translation. The dynamics of the motioncompensation method may be synchronised with the image capture such thatthe undesirable motion of the image is minimised during exposure and thesharpness of the output image is maximised. It is noted that the motioncompensation may shift the image on the sensor which would affect theprincipal point of the camera and may need to be accounted for in imageprocessing, such as bundle adjustment and calibration.

A suitable process for the motion compensation unit 415 of camera 414 isillustrated in the high-level process of FIG. 11 . The system control405 sends signals to control the operation of the motion compensationunit 415, synchronise with the control of the scanning mirror 413, andtrigger the camera 414 to capture motion compensated images with thedesired projected geometry.

The motion compensation unit 415 uses geometry estimator module 450 todetermine the projection geometry 451 of the camera 414 of the scandrive unit 411 in its current configuration that is a function of thescan angle. The projection geometry 451 is the mapping between pixellocations in the sensor and co-ordinates of imaged locations on theground. The co-ordinates on the object area may be the x- and y-axes ofthe various scan pattern illustrations shown in, e.g. FIGS. 4 a and 4 b. The projection geometry 451 may be expressed in terms of a projectivegeometry if the ground is represented as a flat plane, or may use otherrepresentations to handle a more general non-flat object area.

The geometry estimator module 450 may compute the projection geometry451 based on the known scan angle 430 reported in the mirror controldata 437, the known scan drive unit (SDU) geometry data 467, the IMUattitude data 466 that reports the orientation of the scan drive unit,and the aerial vehicle altitude data 464. Optionally, the geometryestimator module 450 may use local ground surface height profile datafrom a Digital Elevation Model (DEM) 465 and latitude/longitude data 463of the aerial vehicle to form a more accurate projection geometry. Thegeometry estimator module 450 may operate at a fixed rate, or may atspecific times for example be based on the settling of the scanningmirror 413 provided through the mirror control data 437.

The projection geometry 451 may be used in combination with variousmotion sensor measurements to estimate pixel velocity estimates. A pixelvelocity estimate is an estimate of the motion of the focused image overthe camera sensor during exposure. Two different pixel velocityestimators are described herein, relating to linear and angular motionof the aerial vehicle. These are referred to as forward motion pixelvelocity estimator 452 and the attitude rate pixel velocity estimator454 respectively.

The forward motion pixel velocity estimator 452 uses the projectiongeometry 451 in addition to the current ground velocity 462 of theaerial vehicle generated by the GNSS receiver 404 to calculate a forwardmotion pixel velocity 453 corresponding to the linear motion of thescanning camera system 408 during the camera exposure. A pixel velocitymay be expressed as an average velocity of the image of the ground overthe camera sensor and may comprise a pair of rates (e.g. expressed inpixels per millisecond), corresponding to the rate of motion of theimage of the ground along the two axes of the sensor. Alternatively, itmay comprise an orientation angle (e.g. in degrees or radians) and amagnitude of motion (e.g. in pixels per millisecond), or any othersuitable vector representation.

The forward motion pixel velocity estimator 452 may compute the forwardmotion pixel velocity 453 by mapping the location on the groundcorresponding to a set of points across the sensor based on theprojection geometry, shifting those points according to the motion ofaerial vehicle over a short time step (e.g. 1 ms or a value related tothe camera exposure time), then projecting back to the sensor. The shiftin each sensor location from the original location due to the motion ofthe aerial vehicle may be divided by the time step to estimate the localvector velocity at the sensor location. The pixel velocity of the imagemay be computed by statistically combining (e.g. averaging) the localvector velocities over the set of sampled sensor location.

The forward motion pixel velocity estimator 452 can operate at a fixedupdate rate, or can operate to update when there are changes to theinput data (ground velocity 462 and projection geometry 451) or based onsome other appropriate criteria.

The attitude rate pixel velocity estimator 454 uses the projectiongeometry 451 in addition to the IMU attitude rates 468 generated by theIMU 409 to calculate an attitude rate pixel velocity 455 correspondingto the rate of change of attitude (e.g. yaw rate) of the scanning camerasystem 408 during a camera exposure. The attitude rate pixel velocity455 may be expressed in the same vector form as the forward motion pixelvelocity 453. The attitude rate pixel velocity estimator 454 may use asimilar short time step based estimation approach to determine theattitude rate pixel velocity 455. A pixel location on the sensor may bemapped to a position on the ground through the projection geometry 451.A second projection geometry is then generated based on the projectiongeometry 451 rotated according to the change in attitude of the scanningcamera system that would occur over the short time step due to thecurrent attitude rate. The position on the ground is mapped back to asensor coordinate based on the second projection geometry. The attituderate pixel velocity 455 may be estimated as the change in sensorposition relative to the original position divided by the time step.

The attitude rate pixel velocity estimator 454 module may operate at afixed update rate, or may operate to update when there are changes tothe input data (IMU attitude rates 468 and projection geometry 451) orbased on some other appropriate criteria. The IMU attitude rates 468 mayhave high frequency components and the attitude rate pixel velocity 455may vary over short times.

It may be advantageous to send multiple updated attitude rate pixelvelocity estimates to the motion compensation control 458 correspondingto a single image capture in terms of the dynamic requirements of themotion compensation drive(s) 460. This is represented in the processflow by the additional ROI pixel velocity estimator 440. It may also beadvantageous to use some kind of forward prediction estimator on the IMUdata to reduce the difference in actual attitude rate between the timeof measurement and the time of the camera exposure. Suitable forwardprediction methods may include various known filters such as linearfilters, Kalman filters and statistical method such as least squaresestimation. The forward prediction methods may be tuned based onpreviously sampled attitude rate data from similar aircraft with similarstabilisation platform and camera system.

In one embodiment, the scanning camera system 408 may be isolated fromroll and pitch rate by a stabilisation platform 407, and the attituderate pixel velocity 455 may be computed based only on the yaw rate ofthe aerial vehicle. In other embodiments the scanning camera system 408may be isolated from roll, pitch and yaw, and the attitude rate pixelvelocity 455 may be assumed to be negligible.

In addition to motion sensor pixel velocity estimators such as theforward motion pixel velocity estimator 452 and attitude rate pixelvelocity estimator 454, a direct measurement of the pixel velocity maybe computed based on captured images. It may be advantageous to performthis analysis on small region of interest (ROI) images 469, preferablytaken in textured regions of the area, in order to reduce the latencybetween the capture of images and the generation of the pixel velocityestimate. The ROI images 469 should be captured in the absence of motioncompensation and may use a short exposure time relative to normal imageframe capture, but preferably after the mirror has settled. The vectorpixel shift may be estimated between ROI images captured at slightlydifferent times using any suitable image alignment method (for examplecorrelation based methods in the Fourier domain or in real space,gradient based shift estimation method, or other techniques). The vectorpixel shift estimate may be converted to a pixel velocity by dividingthe shift by the time step between the time of capture of the ROI image.

The ROI pixel velocity estimator 440 may combine pixel velocityestimates from more than two ROI images to improve accuracy, and it mayoperate with a fixed rate or when ROI images are available. An estimatedROI pixel velocity 457 may be rejected if certain criteria are not met,for example if there is insufficient texture in the images. The locationof the captured images may be set to improve the likelihood of goodtexture being found in the imaged region, for example based on theanalysis of other images captured by the scanning camera system or basedon previous surveys of the same area.

The motion compensation process illustrated in FIG. 11 may be adapted tothe case that one or more scanning mirror structures are not stationaryduring capture. It may be advantageous to allow the mirror to movecontinuously during operation rather than coming to a halt for eachexposure. The alternative process would use an additional scanningmirror pixel velocity estimator that would analyse the motion of thescanning mirror structure during the exposure. The scanning mirror pixelvelocity estimator may use a short time step estimation approach todetermine a scanning mirror pixel velocity. A pixel location on thesensor may be mapped to a position on the ground through the projectiongeometry 451. A second projection geometry is then generated based onthe projection geometry 451 calculated at a second time that is a shorttime after the time of the projection estimate and for a second scanmirror angle corresponding to the expected scan mirror angle at thattime. The position on the ground is mapped back to a sensor coordinatebased on the second projection geometry. The scanning mirror pixelvelocity may be estimated as the change in sensor position relative tothe original position divided by the time step. The scanning mirrorpixel velocity may additionally be supplied to the motion compensationcontrol where it may be combined with the forward motion pixel velocity453 and/or the attitude rate pixel velocity 455.

The motion compensation control 458 combines available pixel velocityestimates that are input to determine an overall pixel velocityestimate, and uses this estimate to control the drives of the motioncompensation unit to trigger the dynamic behaviour of the motioncompensation elements to stabilise the image on the sensor during thecamera exposure time. The motion compensation control 458 also receivestiming signals from the system control 405 that gives the requiredtiming of the motion compensation so that it can be synchronised withthe settling of the scanning mirror structure and the exposure of thecamera. The motion compensation control 458 may optionally use motioncompensation calibration data 461 that may be used to accuratelytransform the estimated overall pixel velocity to be compensated by themotion compensation unit 415 into dynamic information relating to therequired control of the motion compensating elements (for example therotations or tilts of optical plates, mirrors or other components usedin motion compensation).

The attitude rate pixel velocity 455 and forward motion pixel velocity453 estimates are motion sensor based pixel velocity estimates thatcorrespond to different motions of the aerial vehicle. These may becombined by adding together the vector components. Alternatively, asingle estimate may be used for example if only one rate is available,or if one rate is not required (e.g. if the stabilisation platform 407is effectively isolating the scanning camera system 408 from allattitude rates).

The ROI pixel velocity 457 is a directly measured overall pixel velocityestimate that includes the motion from attitude rate and forward motion.The ROI pixel velocity 457 may be used in place of the other pixelvelocity estimates when it is available, or it may be combined with theother estimates statistically (for example based on a Kalman filter orother appropriate linear or non-linear methods).

There may be some latency in the operation of the motion compensationdrive(s) 460 to achieve the appropriate dynamics of the components ofthe motion compensation unit 415. Therefore the motion compensationcontrol 458 can send control signals for the motion of the motioncompensation drive(s) 460 starting at some required time step prior tothe image exposure in order to account for this latency. The motioncompensation control 458 may optionally update the control signals tothe motion compensation drive(s) 460 prior to the image exposure basedon updated pixel velocity estimates such as low latency attitude ratepixel velocity estimator 456. Such low latency updates may be used toachieve a more accurate motion compensation and sharper imagery.

The principle of operation of tilting optical plate motion compensationis based on the refraction of light at the plate surfaces, asillustrated in FIG. 12 . When a light ray 290 is incident on a tiltedoptical plate 291, it is refracted at the front surface 292 according toSnell's law, and then refracted at the rear surface 293 to return to itsoriginal orientation. The effect on the light ray 290 is that it isoffset by a transverse distance δ relative to its original path. Thesize of the offset is proportional to the optical plate's 231 thickness,roughly proportional to the tilt angle (for small angles), and alsodepends on the refractive index of the glass. If the tilt angle (θ_(t))of the optical plate 291 varies with time, then the offset of the rayalso varies. Applying this principle to a camera, varying the tilt of anoptical plate between the lens and sensor may be used to shift the raysof light that focus to form an image on the sensor, thereby shifting theimage on the sensor.

One or more tilting optical plates may be introduced between the cameralens and the sensor. Such plates affect the focus of rays on the sensor,however, this effect may be taken into account in the lens design sothat the MTF of the lens remains high, and sharp images may be obtained.The design is compensated at a design tilt angle of the optical plate,which may be zero tilt, or some nominal tilt related to the expecteddynamics of the plate during exposure. At angles other than the designangle of the optical plate, the change in the optical path results inaberrations and a drop in MTF. For example, dispersion in the glass ofthe optical plate causes rays at different wavelengths to take differentdeviations resulting in some chromatic aberrations and a drop in MTF.This loss of sharpness is small provided that the angle of the platedoes not deviate too much from the design angle.

The optical plates can be manufactured according to tolerances relatingto the flatness of the two surfaces, and the angle of wedge between theopposite surfaces. In one embodiment, they should be built from amaterial with high refractive index and low dispersion. Such glasseswould have a relatively high Abbé number. The plates will be dynamicallycontrolled to follow a desired rotation trajectory; in such a case, aglass with a low specific density and high stiffness can be used. Thetotal thickness and material of optical plates to be placed between thelens and the sensor is a key parameter in the lens design. In oneembodiment BK7 glass may be used as it has good all-round properties interms of refractive index, dispersion, specific density and stiffness,and is also readily available. Other suitable glasses include S-FPL51,S-FPL53, or SPHM-53.

In general, thicker glass plates are better as they require smallertilts to achieve a given motion correction, however the space availablebetween lens and sensor places an upper limit on the plate thickness. Asuitable thickness of glass may be around 10 mm, though it may beunderstood that the methods of motion compensation described in thisspecification are effective over a wide range of glass platethicknesses. Suitable tolerances for the manufacture of the plates maybe surfaces <λ/4 roughness, parallel to <1 arcmin with reflectivity<0.5%.

FIGS. 13 a, 13 b and 13 c illustrate a first arrangement for motioncompensation in the camera of a scanning camera system from aperspective, a side view, and from a view down the optical axis of thelens, respectively. The camera comprises of a focusing lens 240, twooptical plates 241, 242 and a sensor 243. The sensor 243 is mounted inthe appropriate focal plane to capture sharp images of the area. Eachoptical plate 241, 242 is mounted to allow the plate tilt angle to becontrolled about a plate tilt axis. The tilt plate angle may becontrolled using any suitable actuator or rotating motors (such as a DCmotor or brushless motor) coupled by a gearbox, direct coupled or beltdriven.

In FIGS. 13 a, 13 b and 13 c , the tilt axis of the first optical plate241 is orthogonal to the tilt axis of the second plate 242. In thisarrangement the optical plates 241, 242 may be tilted about theirrespective axes to shift the image on the sensor 243 in orthogonaldirections, although non-orthogonal arrangements are possible. An imageof an area may be shifted over the sensor 243 along any vector directionand with a speed that depends on the rates of tilt of the first andsecond optical plates 241, 242. If the image of an area is moving overthe area due to dynamic motions of the camera relative to the area thenthe rates of the two optical plates 241, 242 may be independently set sothat the vector direction of motion and speed act to stabilise theimage.

The transverse shape and size of the optical plates 241, 242 should belarge enough so that all focusing rays of light are incident on thesensor 243. The optical plates 241, 242 may be round, square,rectangular, square bevel or rectangular bevel in shape. One advantageof the rectangular and square based shapes is that they have lowermoment of inertia around the tilt axis, thereby reducing the load on adrive motor used to control the optical plate motion during operation.If the sensor 243 has a non-uniform aspect ratio then the rectangularbased shapes may have a very low moment of inertia while being largeenough to encompass all imaged rays. However, such optical plates dorequire the major axis of the rectangular optical plates 241, 242 to becorrectly aligned with the major axis of the sensor 243. The opticalplates 241, 242 can be mounted so that they may be dynamicallycontrolled to tilt according to required dynamics, as discussed herein.In one embodiment, the optical plates may be 5 mm thick BK7 glass.

FIGS. 14 a, 14 b and 14 c illustrate a second arrangement for motioncompensation in the camera of a scanning camera system from aperspective, a side view, and from a view down the optical axis of thelens, respectively. The camera comprises of a focusing lens 240, asingle optical plate 244 and a sensor 243. The sensor 243 is mounted inthe appropriate focal plane to capture sharp images of the area. Theoptical plate 244 is mounted to allow the plate tilt angle to becontrolled about an arbitrary axis in the plane perpendicular to theoptical axis. This includes tilt around the axis aligned to the sensoraxes (illustrated by rotations 281, 283), and any intermediate angle(such as those illustrated by the rotations 282, 284). An image of anarea may be shifted over the sensor 243 along any vector directiondetermined by the rotation axis and with a speed that depends on therate of tilt of the optical plate 244. If the image of an area is movingover the area due to dynamic motions of the camera relative to the area,then the axis of tilt and the rate of tilt of the optical plate 244 maybe independently set so that the vector direction of motion and speedact to stabilise the image.

The criteria for the transverse shape and size of the optical plate 244are the same as for the optical plates 241, 242, that is to say itshould be large enough so that all focusing rays of light are incidenton the sensor 243. Circular, rectangular, and square shaped plates maybe used. It is noted, however, that since a single plate is used, thespatial restrictions on the plate may be reduced compared to the twinplate case (from FIG. 13 a, 13 b, 13 c ), meaning an increased thicknessof the optical plate 244 may be possible. As discussed above, increasingthe thickness increases the image shift for a given tilt. In oneembodiment the optical plate 244 may be 10 mm thick BK7 glass.

FIGS. 15 a, 15 b and 15 c illustrate another arrangement for motioncompensation in the camera of a scanning camera system from aperspective, a side view, and from a view down the optical axis of thelens, respectively. The camera comprises of a focusing lens 240, twooptical plates 245, 246 and a sensor 243. The sensor 243 is mounted inthe appropriate focal plane to capture sharp images of the area. Eachoptical plate 245, 246 is mounted to with a fixed plate tilt angle asmay be seen in the side view of FIG. 15 b . Each optical plate 245, 246is additionally mounted so that is may be rotated about the optical axiswith a rotation rate and rotation phase that may be controlled. Duringoperation, the two optical plates 245, 246 are rotated withindependently selected rotation rates and independent phases ofrotation. The rotations of the optical plates 245, 246, are controlledsuch that the tilts of the two optical plates 245, 246 are opposed atthe time of exposure of the sensor 243 to capture an image in order tominimise loss of image quality. At the time of exposure, the phases ofthe optical plates 245, 246 determine the vector direction of imagemotion, and the rotation rates of the optical plates 245, 246 determinethe speed of image motion generated by the motion compensation unit ofthe camera. If the image of an area is moving over the area due todynamic motions of the camera relative to the area, then phase androtation rates of the two optical plates 245, 246 may be independentlyset so that the vector direction of motion and speed act to stabilisethe image.

The criteria for the transverse shape and size of the optical plates245, 246 are the same as for optical plates 241,242, that is to say theyshould be large enough so that all focusing rays of light are incidenton the sensor 243. Due to the rotations of the optical plates 245, 246about the optical axes, it may be advantageous to use circular opticalplates. In one embodiment the optical plates 245, 246 may be 5 mm thickBK7 glass tilted at 6°.

Referring back to FIG. 11 , in one embodiment, the motion compensationunit 415 may comprise a pair of optical plates 241, 242, as werediscussed with reference to FIG. 13 a-13 c . Each tilting optical plate241, 242 may be tilted by motion compensation drive(s) 460 according toa trajectory provided by the motion compensation control 458. One ormore motion compensation sensor(s) 459 may be used to track the motionand give feedback to the motion compensation control 458.

FIG. 16 shows some example trajectories suitable for the tilting platemotion. Three sample trajectories are shown, one with a longer latencyT_(lat) ^(A), one with a shorter latency T_(lat) ^(B), and one that isgenerated by adding together a fraction of the longer latency trajectoryand a fraction of the shorter latency trajectory that may be referred toas a mixed latency trajectory, T_(lat) ^(A)/T_(lat) ^(B).

FIG. 16 includes plots of the tilt (top plot), tilt rate (middle plot),and tilt acceleration (bottom plot) associated with the threetrajectories. The plots are each centred around the time (x-axis) 0,which is assumed to be the middle of the image exposure time, and arebased on a piecewise linear tilt acceleration. Alternative trajectoriesmay be formed based on different assumptions such as piecewise constanttilt acceleration, piecewise linear tilt jerk, or other suitableassumptions that may be selected based on the specific motioncompensation control and drive.

The three trajectories of FIG. 16 achieve the same constant tilt rate(zero tilt acceleration) over the time period −T_(exp) to T_(exp) aroundthe time 0. This constant tilt rate time period may be longer than thetotal exposure time of the camera in order to allow for errors in thecontrol of the tilting plate and the timing of the exposure. There maybe some limits on the maximum and minimum tilts allowable, indicated by±θ_(max) in the tilt angle plot. The tilt at time offset of zero (themiddle of the period of constant tilt rate) is zero in order to minimiseloss of sharpness due to non-zero tilt during the exposure.

Comparing the three trajectories, it may be seen that the longer andmixed latency trajectories may be advantageous in terms of theacceleration rates required, while the lower latency may be advantageousin terms of the maximum tilt required. However, if the dynamics of theaircraft have some high frequency components, the mixed and lowerlatency trajectories may be advantageous as they may use more up to datemotion estimates with lower errors over the exposure time.

FIG. 17 a includes 14 object area projection geometries G1 to G14 thatillustrate the 14 frames of the scan pattern of the third scan driveunit 303 of scanning camera system 300 discussed with reference to FIG.3 above. In this instance the scanning camera system 300 is assumed tobe aligned with the motion of the aerial vehicle as may occur in theabsence of yaw. Each ground projection geometry G1-G14 has an arrowrepresenting the forward motion vector of the aerial vehicle. FIG. 17 aalso includes 14 corresponding sensor plots S1 to S14 that illustratesthe corresponding motion compensating pixel velocity relative to thesensor geometry due to forward motion as an arrow in each rectangularsensor outline.

The upper plot of FIG. 17 b shows the components of the motioncompensating pixel velocities illustrated in FIG. 17 a as a function offrame number (1 to 14), where the pixel pitch is 3.2 microns. The lowerplot in FIG. 17 b shows the corresponding plate tilts for the first andsecond optical plates (e.g. optical plate 241, 242) required for motioncompensation. In this case, the plates may be 5 mm BK7 plates, with thefirst axis aligned at 0° and the second at 90° so that tilting the firstplate results in an image shift along the x-axis and tilting the secondplate results in an image shift along the y-axis. The conversion frompixel velocities to plate tilt rates may be achieved using the motioncompensation calibration data, which may consist of thickness, material(refractive index) and orientation data for each of the plates, oralternatively may consist of parameters of functions that may be used toconvert image shifts to plate tilts and vice versa. It is noted thatnone of the pixel velocities of the upper plot of FIG. 17 b include acomponent in the x-axis and therefore the tilt rate for the first plateis zero for all frames. In this particular case the first plate isredundant.

FIG. 18 a includes 26 object area projection geometries G1 to G26 thatillustrate the 26 frames of the scan pattern of the first scan driveunit 301 of scanning camera system 300 discussed with reference to FIG.4 a-4 f above. The scanning camera system 300 is assumed to be alignedwith the motion of the aerial vehicle and each ground projectiongeometry has an arrow representing the forward motion vector of theaerial vehicle. FIG. 18 a also includes 26 corresponding sensor plots S1to S26 that illustrates the corresponding motion compensating pixelvelocity relative to the sensor geometry due to forward motion as anarrow in each rectangular sensor outline.

FIG. 18 b gives plots of the pixel velocity components (where the pixelpitch is 3.2 microns) of the frames illustrated in FIG. 18 a and thecorresponding tilt rates of the first and second plates required formotion compensation, again assuming 5 mm BK7 plates, with the first axisaligned at 0° and the second at 90°. Due to the scan pattern of thefirst scan drive unit 301, the pixel velocities generally have non-zerocomponents along both axes and therefore both optical plates are used.

FIG. 19 a shows a tilt trajectory for the first optical plate that maybe used to achieve motion compensation for the required tilt rates shownin the second, lower plot of FIG. 18 b . The trajectory consists of 26sections that are scaled copies of the longer latency trajectory of FIG.16 joined by stationary sections of zero plate tilt. The scaling of eachsection is set according to the required tilt rates of the first opticalplate. Alternative trajectories may be formed based on the shorterlatency trajectory of FIG. 16 or a mixed latency trajectory, or may usea mixture of trajectories with different latencies or mixtures oflatencies. FIG. 19 b shows a tilt trajectory for the second opticalplate that may be used to achieve motion compensation for the requiredtilt rates shown in the second, lower plot of FIG. 18 b . Thistrajectory was formed in the same way as the tilt trajectory for thefirst optical plate shown in FIG. 19 a . In the plots shown in FIGS. 19a and 19 b , increments between each pair of adjacent dashed verticallines along the x-axis equates to 75 milliseconds.

FIGS. 20 a and 20 b illustrate how alignment of the optical platesaffects the computed motion compensation tilt rates through the motioncompensation calibration data. FIG. 20 a shows an alternative set ofmotion compensation plate tilt rates computed for the first scan driveunit 301 and for the same pixel velocity data as FIG. 18 b , but for 5mm BK7 plates oriented at 45° and 135°. FIG. 20 b shows an alternativeset of motion compensation plate tilt rates computed for the second scandrive unit 302 and for the same pixel velocity data as FIG. 18 b , butfor 5 mm BK7 plates oriented at 45° and 135°.

FIGS. 21 a and 21 b illustrates how the pixel (pitch: 3.2 microns)velocities and tilt rates are affected by the alignment of the scanningcamera system 300 relative to the flight path, specifically for the caseof a 15 degree yaw that is not corrected in the stabilisation platform.FIGS. 21 a and 21 b show the pixel velocities and tilt rates for scandrive unit 301 and scan drive unit 302 respectively, and for the case of5 mm BK7 tilting plates oriented at 0° and 90°, respectively.

FIGS. 22 a and 22 b illustrate how the pixel (pitch: 3.2 microns)velocities and tilt rates are affected by the rate of change of attitudeof the scanning camera system 300, specifically for the case of yawrates of up to 3° per second, randomly sampled at each frame and notcorrected in the stabilisation platform. FIGS. 22 a and 22 b show thepixel velocities and tilt rates for scan drive unit 301 and scan driveunit 302 respectively, and for the case of 5 mm BK7 tilting platesoriented at 0° and 90°, respectively.

FIGS. 23 a and 23 b illustrates how the pixel (pitch 3.2 microns)velocities and tilt rates are affected by the rate of change of attitudeand alignment of the scanning camera system 300 relative to the flightpath, specifically for the case of a yaw of 15° and a yaw rate of up to3° per second that is not corrected in the stabilisation platform and israndomly sampled at each frame. FIGS. 23 a and 23 b show the pixelvelocities and tilt rates for scan drive unit 301 and scan drive unit302 respectively, and for the case of 5 mm BK7 tilting plates orientedat 0° and 90° respectively.

Similar techniques to those applied to generate the sample trajectoriesof FIGS. 17 a, 17 b, 18 a, 18 b, 19 a, 19 b, 20 a, 20 b 21 a, 21 b, 22 a, 22 b, 23 a and 23 b may also be applied to the single tilting opticalplate case of FIG. 14 . In this case, however, there would be a singleplate (i.e. optical plate 244) of roughly double the thickness of asingle plate (e.g. 10 mm BK7) and the tilting plate drive would beactuated to achieve a tilt rate and a tilt orientation. The tiltorientation would be computed based on trigonometric operations on thex- and y-components of the pixel velocity, while the tilt magnitudewould be computed based on the magnitude of the pixel velocity vector.

The computation of spin rates and phases for the spinning tilted platemotion compensation unit discussed with reference to FIGS. 15 a, 15 band 15 c is more complicated. The two plates (i.e. optical plates 245,246) should be controlled to spin in opposite directions such that atthe middle of the exposure time they are oriented with an opposed tilt.The opposite tilt should be oriented according to the vector directionof the required pixel velocity, and equal and opposite spin rates shouldbe used for the plates with a magnitude determined in accordance withthe plate thicknesses, plate materials and the required pixel velocitymagnitude. Such a trajectory may be achieved by using a similartrajectory to that shown in FIG. 16 , however such a trajectory mayrequire very large drive torque and it may be more efficient to use acontinuous spinning operation for certain frames depending on the motioncompensation pixel velocity requirements. In one embodiment, the opticalplates may be 5 mm thick BK7 glass tilted at 6°.

In the case that the motion compensation requirements are mostly due tolinear motion of the aerial vehicle, the errors in motion compensationthat arise from the variable projection geometry over the sensor pixelsmay be reduced by introducing a small angle between the sides of one orboth optical plate (i.e. a wedge) in the tilting plate cases. In thecase that the motion compensation requirements include a significantcontribution from the attitude rate pixel velocity, any advantage ofthis wedge configuration would be reduced.

An alternative view of the scanning camera system 300 is shown in FIG.24 that is based on a solid model of the camera system components fixedinto a stabilisation platform 407. From above, the mirror structures aremostly occluded by the mounting structures that hold the camera systemcomponents in place. FIGS. 25, 26, 27, 28 and 29 illustrate how theaerial vehicle's attitude affects the orientation of the scanning camerasystem 300 in a stabilisation platform 407.

FIG. 25 shows a top and bottom view of the scanning camera system 300for the case of an aerial vehicle aligned with the flight lines(y-axis), as might be the case for the aerial vehicle flying in theabsence of roll, pitch or yaw. The survey hole 305 is aligned with theaerial vehicle, and therefore also with the flight lines. The scanningcamera system 300 can be seen to fit in the survey hole 305 with a smallmargin around the perimeter.

FIG. 26 shows a top and bottom view of the scanning camera system 300for the case that the aerial vehicle is aligned with the flight lines(y-axis) with a roll of 6° that has been corrected by the stabilisationplatform 407. This configuration is equivalent to survey hole 305remaining aligned with the flight lines but rotated around the axis ofthe flight lines relative to the scanning camera system 300. The marginaround the perimeter of the survey hole 305 is slightly reduced due tothe roll.

FIG. 27 shows a top and bottom view of the scanning camera system 300for the case that the aerial vehicle is aligned with the flight lines(along the y-axis) with a pitch of 6° that has been corrected by thestabilisation platform 407. As was for the case of roll shown in FIG. 26, the margin around the perimeter of the survey hole 305 is slightlyreduced.

FIG. 28 shows a top and bottom view of the scanning camera system 300for the case that the aerial vehicle is aligned with the flight lines(y-axis) with a yaw of 15° that has been corrected by the stabilisationplatform 407. The larger of yaw (15°) modelled is selected to berepresentative of the range of dynamics that may be seen in the range ofcommercial aerial vehicles in which the scanning camera system 300 maybe deployed. In contrast to the roll and pitch cases of FIGS. 26 and 27, the margin around the perimeter of the survey hole 305 is greatlyreduced, so that the scanning camera system 300 may no longer fit in thesurvey hole 305.

In order to reduce the spatial requirements in the survey hole 305, thestabilisation system may be configured to correct only for roll andpitch. This conveys the added advantage of reducing the size, cost andcomplexity of the stabilisation platform 407. FIG. 29 shows a top andbottom view of the scanning camera system 300 for the case that theaerial vehicle is aligned with the flight lines (y-axis) with a yaw of15° that has not been corrected by the stabilisation platform 407. Theconfiguration of the scanning camera system 300 relative to thestabilisation platform 407 is identical to that shown in FIG. 25 ,however the scanning camera system 300 is rotated according to the yawso that the captured scan patterns are rotated on the object area. In anembodiment, the scan angle can be set based on a difference between theyaw angle of the vehicle and a preferred yaw angle (e.g. zero). The scanangle can be adjusted during or between one or more flight lights.

FIG. 30 a illustrates the scan patterns on the ground for the scanningcamera system 300 when the aerial vehicle has a yaw of 15° relative tothe flight line (y-axis). The curved and linear scan patterns that makeup the overall system scan pattern are all rotated by the yaw anglearound the z-axis. Images captured with these rotated scan patterns mayhave lower quality relative to those captured without the yaw as seen inFIG. 1 a . The drop in quality may be correspond to loss of coverage ofspecific azimuthal angles of oblique imagery (e.g. increased tolerancein captured imagery relative to the cardinal directions), a slightincrease in the maximum obliqueness of the vertical imagery due to theangle of the linear scan pattern through the vertical, and/or otherfactors. FIG. 30 b illustrates three sets of scan patterns with forwardoverlaps that may be captured during the operation of a scanning camerasystem in an aerial vehicle with a yaw of 15°.

One aspect of the present disclosure is the design of the first scandrive unit 301 that captures oblique images. The selection of scanangles within a scan pattern may be advantageously modified in order tocorrect for the yaw of the aerial vehicle. Specifically, a correction ofone half of the yaw applied to each sampled scan angle of the scanningmirror can be used to generate a scan pattern that is the same as thescan pattern that would have been generated in the absence of yaw withthe original scan angles. FIG. 31 shows a top and bottom view of thescanning camera system 300 for a case that the aerial vehicle is alignedwith the flight lines (along the y-axis) with a yaw of 15° that has beencorrected by an offset scan angle of the scanning mirror (that is acorrection of 7.5° of the scanning mirror scan angle relative to thescanning mirror of FIGS. 25 to 29 ).

FIG. 32 a illustrates the scan patterns on the ground for the scanningcamera system 300 when the aerial vehicle has a yaw of 15° relative tothe flight line (y-axis) with scan angle yaw correction performed in thefirst scan drive unit 301. The curved scan patterns corresponding to thefirst scan drive unit 301 match those of FIG. 1 (without yaw), while thelinear scan patterns corresponding to scan drive unit 302 and scan driveunit 303 are rotated by the yaw angle around the z-axis. In this casethe drop in quality of oblique imagery is eliminated, while the smallloss in image quality due to the slight increase in vertical imagerymaximum obliqueness discussed above remains. The overall quality of thegenerated images has therefore improved through the yaw correctionprocess based on the adaptive control of the scan angles of the firstscan drive unit 301. FIG. 32 b illustrates three sets of scan patternswith forward overlaps that may be captured during an operation of thescanning camera system in an aerial vehicle under the configurationdescribed with respect to FIG. 32 a.

The range of scan angles of the first scan drive unit 301 required tohandle yaws between −15° and 15° is larger than the range of scan anglesused for imaging in the absence of yaw. Specifically, the range of scanangles is extended by 7.5° in each direction from the standard range(−30.7° to +30.7°) to give an extended range (−38.2° to +38.2°). Thestandard mirror geometries designed for the standard scan angle rangediscussed with reference to FIG. 4 e would not be large enough to handlescan angles beyond the standard range. If a mirror is set to a scanangle beyond its design range then light from light beams originating inother locations in the area can pass around the outside of the mirrorrather than reflecting from the mirror. This light is incident on thelens and focused on the sensor resulting in ghost images in the capturedimages (images of another area superimposed on the captured image).

FIGS. 33 a and 33 b help to illustrate the formation of a ghost imagedue to a mirror that was designed for a smaller range of scan anglesthan the current scan angle setting. FIG. 33 a shows a camera 250 thatis imaging an area 251 reflected in a mirror 252. The camera 250 islocated inside a survey hole 253 and the imaged area 251 is very closeto the camera 250, however the principle demonstrated in FIG. 33 a maybe generalised to an area at a much greater distance from the camera 250as would be the case in an aerial survey. The light from location 254,imaged by the camera 250, forms a beam 255 that is focused on a sensorin camera 250 at a particular pixel that corresponds to the point on theground at location 254. FIG. 33 b shows the same arrangement, howeverthe mirror 252 from FIG. 33 a is replaced by a smaller mirror 256 aroundwhich a second beam 257 from a second location 258 in the area 251passes. The second beam 257 is focused by the camera lens to the samepixel location on the sensor of the camera 250 as a third beam 259, thatis the subset of the first beam 255 in FIG. 33 a defined by the reducedmirror geometry.

Extending the illustration of FIG. 33 b , each pixel in the sensor maybe exposed to some light from a reflected beam, such as beam 259, and tonon-reflected light from a beam, such as beam 257. The exposure of thesensor therefore includes a reflected image component due to reflectedbeams of light and a ghost image component due to direct image beamsthat pass around the mirror. Furthermore, the reflected image componentmay have a reduced exposure compared to the case that the mirror issufficiently large to handle all beams focused onto the sensor, and thatreduced exposure may vary across the sensor (vignetting).

FIG. 4 f illustrated an extended mirror geometry computed for the caseof over-rotation (“over”), that is for the extended rotation range thatwould be appropriate to capture the curved paths of the scan pattern ofFIG. 32 a without ghost image formation. The extended scanning mirrorgeometry is larger than the standard mirror geometries of FIG. 4 e thatwere designed for the standard scan angle range. In some instances, thecost and complexity of manufacturing the extended scanning mirror can beincreased relative to a standard scanning mirror due to its increasedsize. Furthermore, the mass and moment of inertia of an extended mirrorcan be greater than a standard scanning mirror so that the dynamicperformance of the extended mirror may be reduced, and the cost andcomplexity of mounting and controlling its movements may be increased.

In one embodiment of the present disclosure, increased costs, complexityand reduced dynamic performance of the extended mirror may be mitigatedthrough the use of a hybrid mirror structure. A hybrid mirror structureis based on a standard mirror structure extended out to the geometry ofthe extended mirror using sections of lightweight low reflectivitymaterial. The key advantage of the hybrid mirror is that lowreflectivity material sections block unwanted light beams consisting ofrays of light that would otherwise pass around the mirror scan anglesbeyond the standard range, thereby preventing loss of quality due to theassociated ghost images. The lightweight extensions also result in alower moment of inertia when compared to a full extended scanningmirror, such that the dynamic performance is increased.

FIG. 34 a shows an illustration of the hybrid mirror in a scan driveunit 301 according to an embodiment of the invention. The low-reflectivematerial 317 is added around the scanning mirror structure 312 toimprove image quality when the scan angle is beyond the standard range.

FIG. 34 b illustrates the principle of operation of the hybrid mirror toprevent ghost images for the arrangement shown in FIG. 33 b . The mirror256 has been modified by the addition of a section of low-reflectivematerial 260 that blocks the beam 257 from the second location 258 thatwould contribute to a ghost image. The added low-reflective material 260does not reflect the light beam 261 from the ground point location 254that is a subset of the original beam 255 of FIG. 33 a . The beam 259that is also a subset of beam 255 is, however, reflected from thereflective surface of the mirror 256 and focused through the camera lensonto the camera's 250 sensor. The surface quality of the reflectivesurface of the mirror 259 needs to be sufficiently high in order togenerate a high quality focused image that may be captured by thesensor. In this way the ground location 254 is imaged, however theground location 258 that is associated with a ghost image, is notimaged. On the other hand, since there is no specular reflection fromthe low-reflective material 260, the surface quality (roughness,flatness, reflectivity) does not need to be high in order to maintainthe overall sharpness and quality of images captured on the sensor.

The exposure of the pixel corresponding to the area location 254 isreduced since only a subset (i.e. beam 259) of the original beam 255 isreflected by the mirror 256 and focused onto the sensor. The exposure ofother pixels on the sensor may be reduced to a greater or lesser extentdue to the mirror geometry being smaller than required. This results ina form of vignetting where the exposure is a function of location on thesensor, and a captured image may look darker over some regions comparedto others. The vignetting will be discussed further below with respectto FIGS. 36 a and 36 b . This vignetting may be modelled and correctedas will be discussed further below.

The low reflectivity material can be attached to the mirror in a secure,stiff manner such that it moves with the mirror structure blockingunwanted beams. Given that the sections no longer need to meet tightoptical specifications in terms of flatness and reflectivity they may bemanufactured from lightweight low-cost materials, for examplecarbon-fibre. This conveys the additional benefit of reducing the momentof inertia and mass of the hybrid mirror structure relative to anextended mirror structure. The reduced moment of inertia and mass of themirror structure may allow for faster rotation of the scanning mirrorbetween requested scan angles, and therefore a faster scanning camerasystem. The low reflectance material sections may change the overallgeometry of the hybrid mirror structure relative to the standard mirrorstructure. For example, they may form non-convex extensions to a convexstandard mirror structure.

In another embodiment of the present disclosure, the aperture of thecamera may be dynamically tuned such that the geometry of the mirrorsurfaces 314, 315 of scanning mirror structure 312 are large enough toreflect all rays that are focused onto the sensor. Specifically, theaperture is reduced as the scan angle extends beyond the designparameters of the mirror (i.e. when over-rotation occurs). In oneembodiment the aperture may be reduced symmetrically. In otherembodiments the aperture may be reduced asymmetrically. The asymmetry ofthe aperture may be selected to minimise the change in aperture whileremoving all beams associated with ghost images. This can minimise theloss of exposure over the sensor. The smallest required asymmetricchange in aperture may take an arbitrary shape. Another approach is touse a simple dynamic change to the aperture, such as one or more slidingsection of opaque material each of which is moved to close the aperturefrom a particular side so as to selectively block some part of theaperture. This may be achieved using a modified, possibly, asymmetriciris to control the aperture. Alternatively an active element such as anLCD may be used to create a dynamic aperture that may be controlledelectronically to form a wider variety of shapes up to the resolution ofthe element. An active aperture may give greater control over theaperture and a faster speed of update compared to sliding sections ofmaterial. On the other hand it may be less practical and may notconstitute as effective a block, with the risk of a small fraction beingtransmitted through the aperture.

As was discussed with reference to FIGS. 25, 26, 27, 28 and 29 , thegeometry of the survey hole can be a constraint in the design of ascanning camera system suitable of deployment in an aerial vehicle. Thecomponents of the scanning camera system must be mounted inside thesurvey hole. Furthermore, if a stabilisation platform is used tomaintain the attitude of the scanning camera system during flight thenthere should be sufficient margin spatially for the scanning camerasystem to rotate with the stabilisation platform without touching thesurvey hole walls.

Further to this spatial constraint, there is an optical constraintrelating to the placement of the scanning camera system in the surveyhole that is illustrated using FIGS. 35 a and 35 b . FIG. 35 a shows thecamera 250 imaging the location 254 of the area 251, reflected in themirror 252, after the survey hole 253 has moved relative to the camera250 and mirror 252. This situation might occur in the case that thecamera 250 and mirror 252 are mounted on a stabilisation system on thesurvey hole 253, and the survey hole 253 attitude is changed, forexample through a roll or pitch of the aerial vehicle that it isattached to. In this case the beam 255 of light consists of two parts:(1) the first part of the beam 262 reflects from the mirror 252 and isfocused onto the sensor by the camera lens, and (2) the second part ofthe beam 263 is occluded by the survey hole 253 and does not reflectfrom the mirror 252 to be focused onto the sensor.

The pixel corresponding to the area location 254 is exposed less due tothe occlusion. The exposure of other pixels on the sensor may be reducedto a greater or lesser extent due to the occlusion. This results in aform of vignetting where the exposure is a function of location on thesensor, and a captured image may look darker over some regions comparedto others.

It is noted that some parts of the full beam 255 may be occluded by thesurvey hole so that they are not incident on the low-reflective mirrorsections. This is illustrated in FIG. 35 b , in which a beam 263 isoccluded by the survey hole 253 and therefore does not reach thelow-reflective material 265 attached to the mirror 266.

The vignetting of images due to the geometries represented in FIGS. 34b, 35 a and 35 b is further illustrated by FIGS. 36 a through 36 h .FIGS. 36 a to 36 h illustrate the calculation of vignetting and ghostimages due to the geometry of the scan drive unit in a survey hole,optionally mounted on a stabilisation platform. The calculations arebased on projecting geometry of various components and objects along theimage beam path onto the aperture plane of the camera assuming multiplesensor locations. This calculation of projection geometry illustrates amodel of the illumination of an image sensor of a camera by an imagingbeam, according to one embodiment. The model of the illumination takesinto consideration factors such as a geometry of a constrained spacehousing a scanning camera system, scan angle of a scanning mirrorstructure, geometry of the scanning mirror structure, and roll/pitch/yawof a vehicle housing the scanning camera system to model theillumination of an image sensor in a camera by an imaging beam.

FIG. 36 a shows an image of a uniform untextured surface that isaffected by vignetting. The darker parts of the image (e.g. sensorlocation 277) are more strongly affected by the vignetting than thelighter parts of the image (e.g. location 273).

Nine sensor locations 271, 272, 273, 274, 275, 276, 277, 278, 279 inFIG. 36 a are indicated, and the vignetting of the image at each sensorlocation is illustrated further in the corresponding plots of FIG. 36 b. Each plot of FIG. 36 b illustrates the illumination of the aperture bylight reflected from the mirror of a scan drive unit. The centre of eachplot in 36 b represents the intersection of the optical axis of the lenswith the aperture plane. The solid circular line represents theaperture, while the dashed contour represents the projection of themirror surface geometry onto the space of the aperture. If the dashedcontour extends to or beyond the solid circle, then the mirror issufficiently large for the camera aperture. Any part of the circle notinside the dashed contour is, however, not illuminated by the reflectedbeam from the mirror. The dotted line is part of a larger contour thatrepresents the survey hole. Within the plots, the survey hole is to theleft of the dashed line, so that any part of the solid circle to theright of the aperture is not illuminated by reflected light from themirror due to occlusion by the survey hole. The diagonal hashed part ofthe solid circle represents the fraction of the aperture that isilluminated by reflected light from the mirror, which may be related tothe exposure of the sensor pixel corresponding the plot. It is seen thatthe degree of vignetting varies across the sensor and may depend on bothsurvey hole occlusion and the finite mirror geometry.

A vignetting image for a uniformed untextured area may be formed asdiscussed above with respect to FIGS. 36 a and 36 b . The vignettingimage may be generated at the full sensor resolution, or at a lowerresolution, in which case the vignetting at any given pixel may beestimated by interpolating the vignetting image. The vignetting imagemay be stored as vignetting data 473 in data storage 406. Thisvignetting data 473 can be used to update pixels values to compensatefor vignetting, according to one embodiment.

FIG. 36 b further illustrates the requirements for dynamically tuningthe aperture of the lens to avoid ghost imaging. Specifically, any partof the circular aperture that is not contained within the dashed linecorresponding to the projected mirror geometry should be masked by thedynamic aperture mask. This defines a minimum level of masking, and asdiscussed above, it may be more practical to mask a larger or moreregular region.

FIG. 36 c illustrates an image that may be captured for the samegeometry represented in FIGS. 34 b, 35 a and 35 b but with a modifiedaperture. The variation in illumination is substantially eliminated, sothat the image should no longer be affected by vignetting or a ghostimage.

FIG. 36 d illustrates an irregular and asymmetric region that defines amodified aperture that may be achieved by dynamically reducing thecircular aperture of FIG. 36 b . The full irregular region is hashed atall sensor locations, indicating that the geometry of the systemincluding the survey hole and mirror has not affected the exposure ofthe sensor. This substantially removes the vignetting and ghost imagesthat result from the geometry. As was the case for FIG. 36 b , thecentre of each plot in 36 d represents the intersection of the opticalaxis of the lens with the aperture plane. The same is true for each plotin 36 e, 36 f, 36 g and 36 h.

FIG. 36 e illustrates a first alternative irregular region that definesa modified aperture that may be achieved by dynamically reducing thecircular aperture of FIG. 36 b . Specifically the circularly symmetricaperture is modified by blocking a segment defined by drawing a singlestraight line across the circle. Most of the irregular region of FIG. 36e is hashed in most images, though there is a small part that is nothashed in sensor locations (e.g. 271, 273, 276 and 279). These smallregions would introduce a small amount of vignetting and may also allowfor ghost images if the mirror does not have low reflectance materialextensions that block ghost images.

FIG. 36 f illustrates a second alternative irregular region that definesa modified aperture that may be achieved by dynamically reducing thecircular aperture of FIG. 36 b . Specifically the circularly symmetricaperture is modified by blocking three segments, each defined by drawinga single straight line across the circle. The full irregular region ishashed at all sensor locations, indicating that the geometry of thesystem including the survey hole and mirror has not affected theexposure of the sensor. This substantially removes the vignetting andghost images that result from the geometry.

FIG. 36 g illustrates the aperture plane geometry for a similar case tothat shown in FIG. 36 b but with the scanning mirror angle modified suchthat the mirror geometry projection is deformed, and such that thesurvey hole does not block any of the image beams that are incident onthe full aperture. Most of the irregular region of FIG. 36 e is hashedin most images, though there is a small part that is not hashed insensor locations (e.g. 271, 273, 274, 276 and 277). These small regionswould introduce a small amount of vignetting and may also allow forghost images if the mirror does not have low reflectance materialextensions that block ghost images.

FIG. 36 h illustrates a third alternative region that defines a modifiedaperture that may be achieved by dynamically reducing the circularaperture of FIG. 36 b symmetrically resulting in a smaller circularaperture. The full region is hashed at all sensor locations, indicatingthat the geometry of the system including the survey hole and mirror hasnot affected the exposure of the sensor. This substantially removes thevignetting and ghost images that result from the geometry.

System control 405 receives the IMU attitude data (roll, pitch, and/oryaw) and the scan drive unit parameters 434 including the scan angles.System control 405 is programmed to correlate the IMU attitude data andthe scan angles with the presence of occlusion due to, for example, thesurvey hole 253, and the aperture not being contained within theprojected mirror geometry to compute dynamic aperture settings for agiven frame. System control 405 may compute the dynamic aperturesettings on the fly, the computation being based on parameters such asthe geometry of the scanning camera system, the scan drive angle, thegeometry of occluding objects such as the constrained camera hole,parameters of the camera such as sensor geometry and focal length, andflight parameters such as roll, pitch and yaw. Alternatively, it may usepre-defined look up tables of dynamic aperture parameters that may befunctions of parameters such as scan angle and/or the roll, pitch and/oryaw of the aircraft. System control 405 controls the dynamic aperturethrough signals sent to the cameras, illustrated as 414 and 416 in FIG.10 . Based on the control signals, the aperture may be modified eithermechanically (e.g. through the motion of one or more iris elements) orelectronically (e.g. for an LCD aperture) or otherwise. In anembodiment, the aperture can be modified using one or more motors (e.g.stepper motor, DC motor). The aperture can be reduced symmetrically, forexample as shown in FIG. 36 h , asymmetrically, for example as shown inFIGS. 36 b and 36 f , or a combination of the two, for example as shownin FIG. 36 d.

FIG. 37 illustrates post-processing analysis that may be performed afterimages have been captured for a given aerial survey. The post-processinganalysis may be performed in flight or after the flight, and may beperformed on a computing platform such as a computer or a cloudprocessing platform. The analysis uses data from the data storage 406which may be copied to other data storage after or during the flight. Inone embodiment, the post-processing analysis can be performed using anetwork controller, such as an Intel Ethernet PRO network interface cardfrom Intel Corporation of America, for interfacing with a network. Ascan be appreciated, the network can be a public network, such as theInternet, or a private network such as an LAN or WAN network, or anycombination thereof and can also include PSTN or ISDN sub-networks. Thenetwork can be wired, such as via an Ethernet network, or can bewireless, such as via a cellular network including EDGE, 3G, 4G, and 5Gwireless cellular systems. The wireless network can also be Wi-Fi,Bluetooth, NFC, radio frequency identification device, or any otherwireless form of communication that is known.

One or more individual captured images may optionally be processed by avignetting analysis process 474 to generate vignetting data 473 that maybe used to correct for vignetting of image pixels due to occlusion bythe survey hole 305 or due to the finite geometry of the scanning mirrorstructure of a scan drive unit. The vignetting analysis process 474 maybe performed as was discussed above with reference to FIGS. 36 a and 36b . It may use the SDU geometry data 467, the mirror control data 437and gimbal angles 470 corresponding to a given image from the pixel data439. It may additionally use data defining the survey hole geometry 471,and mirror data 472 relating to the geometry of a scanning mirrorstructure in order to determine the fractional exposure of the apertureas illustrated in FIG. 36 b for multiple pixels in the sensor and thento generate a vignetting image as discussed above.

In one embodiment, the exposure data for specific pixels is stored as afractional exposure, where the fractional area is the fraction of thecircular region corresponding to the aperture is filled with thediagonal cross hatch. A fractional exposure of 1 would represent a fullexposure corresponding to the case that the circular region in FIG. 36 bis fully filled by the diagonal hatch region. The vignetting image mayconsist of fractional exposure data corresponding to specific pixels andmay be stored as vignetting data 473. The vignetting data 473 may beused to correct individual pixels from the pixel data 439 by modifyingthe pixel values according to the vignetting data 473 for that pixel.For example, a pixel RGB value may be divided by the fractional exposurecorresponding to that pixel stored in the vignetting data. Thevignetting data 473 may be interpolated to provide suitable vignettingdata for all pixels in the image. In another embodiment the fractionalexposure may be weighted according to the angle of incidence of rays onthe aperture, for example through a cosine or other trigonometricfunction.

The post-processing of pixel data illustrated in FIG. 37 begins atprocessing step 475 which estimates the pose and position of the cameracorresponding to each image in a global coordinate system. This pose andposition may correspond to a virtual camera that represents the apparentviewpoint and view direction of the camera (i.e. under the assumptionthat no mirrors were in the optical path at the time of image capture).Processing step 475 may use standard known techniques sometimes referredto as bundle adjustment and may use pixel data 439 from one or morefixed overview cameras in addition to the scanning camera system.Processing step 475 may use various survey data corresponding to thecaptured images including latitude/longitude data 463, altitude data464, IMU attitude data 466, motion compensation data 435, mirror controldata 437, and SDU geometry data 467. Processing step 475 may optionallygenerate additional data related to nonlinearities of the cameras (e.g.barrel distortion) and other aspects of the imaging system componentsand the environment in which the images were captured (e.g. atmosphericeffects).

Processing step 475 may optionally be followed by a refinement step 476that improves the various estimates or poses, position and other aspectsof the imaging system and/or environment. The camera poses, positionsand additional data 477 are stored for use in generating various imageproducts based on the survey.

A process for 3D surface reconstruction 478 may use the camera poses,positions and additional data 477 plus pixel data 439 to generate a 3Dtextured surface using known techniques that are described elsewhere. 3Dsurface reconstruction 478 may optionally use vignetting data 473 toimprove the quality of the output by correcting for vignetting in thecaptured images by updating pixel values using a model of illuminationof the image sensor by the imaging beam.

A process for orthomosaic generation 479 may use the camera poses,positions and additional data 477 plus pixel data 439 to generate anorthomosaic 482 using known techniques that are described elsewhereherein. Orthomosaic generation 479 may optionally use vignetting data473 to improve the quality of the output by correcting for vignetting inthe captured images.

A process for vignetting compensation 480 may use the camera poses,positions and additional data 477 plus pixel data 439 and vignettingdata 473 to generate raw imagery that has been corrected for vignettingin the captured images.

In some embodiments, the captured images may be cropped, or region ofinterest imaging may be employed such that the captured frames used forthe analysis described with respect to FIG. 37 may have a variety ofdifferent pixel dimensions. There may be a number of advantages to thisapproach such as reducing the data storage requirements of capturedimage pixels and also removing pixels with lower quality due tovignetting from generated image products.

By capturing images at scan angles such that the captured images haveoverlapping portions, portions of the images can be stitched together toform a cohesive image even after other portions of the image affected byvignetting have been cropped out.

The cropping can include removing some or all portions affected byvignetting. The scan angles can be chosen based on a model of theillumination of the image sensor by the imaging beam, where theillumination may be reduced by partial occlusion from a constrainedspace, the scanning mirror structure being outside a predetermined rangeof scan angles, or a combination thereof. In one embodiment, thepredetermined range of scan angle is determined by the mirror geometry.For example, the regions discussed with respect to FIGS. 36 a to 36 hcan be used to model the illumination of the image sensor by the imagingbeam to know the image sensor locations that are and are not affected byvignetting. For those portions that have vignetting, steps of the scanangles can be smaller to obtain images with enough overlap. In otherwords, different step sizes for the scan angle can be used for differentranges of scan angles. In an embodiment, a step size of the values ofthe scan angle of the scanning mirror structure based upon on at leastone of: a yaw angle of a vehicle including the imaging system; a roll ofthe vehicle; a pitch of the vehicle; a geometry of the scanning mirrorstructure; the scan angle; and a geometry of the constrained space.

FIG. 38 a illustrates the projective geometry of a suitable set ofcropped image frames for the scanning camera system 300 and for two scanpatterns along the flight path. The overlap of the projection geometryof frames along the curved paths of scan pattern 111, 112 is seen to bemore uniform than was seen in FIG. 1 b and this has been achieved bycropping the sensor pixels associated with the outer edge of the curvedpaths for scan pattern 111, 112. In this case the cropped pixels arefound either at the top or bottom assuming a landscape orientation ofthe sensor. The outer, cropped pixels with higher obliqueness, aregenerally more affected by vignetting due to the outer edge of thesurvey hole and therefore there is an advantage to rejecting thesepixels and preserving higher quality pixels taken from the sensorpositions corresponding to the inner geometry of the curved paths forscan pattern 111, 112 and lower obliqueness.

In some cases, it may additionally be advantageous to capture images ata higher rate so that the forward overlap of scan patterns is increased.The increased forward overlap may allow for rejection of an increasedset of pixels along the exterior of scan patterns 111, 112 withoutcompromising the overlap of pixels that may be required forphotogrammetry and image post-processing.

It may further be advantageous to crop pixels of scan patterns 111, 112on the sides of the sensor rather than just the top or bottom. Forexample, in the case that mirror over-rotation is used to achieve yawcorrection it may be advantageous to crop pixels on one or both sides ofthe sensor. The location and number of cropped pixels may be selectedbased on vignetting due to the survey hole or low-reflective sectionsattached to the exterior of the scanning mirror.

Cropping pixels on the sides of the sensor may reduce the overlap ofadjacent image pixels, however the required overlap may be recovered byincreasing the sampling of scan angles of the scanning mirror used inparts of the scan pattern corresponding to frames to be cropped. This isillustrated in FIG. 38 b , where the spacing of projected geometry offrames is seen to be reduced towards the frames 125,126 of scan patterns111, 112 respectively due to cropping the sides of images. The number offrames has, however, been increased so that the required overlap ismaintained between adjacent frames (in this case 10%). The spacing ofthe samples may vary according to any suitable criteria. The spacing mayalternate between discrete values at particular threshold values of scanangle, for example it may be defined by a larger spacing over aparticular range of scan angle and by a smaller spacing beyond thatrange of scan angle. The particular range of scan angles may correspondto the range of scan angles for which a scanning mirror geometry wasdetermined. Alternatively the spacing may vary according to a functionof the scan drive angle. In one embodiment the function may be based ontrigonometric functions over particular ranges of the scan angle. Othersuitable functional forms may be defined based on polynomial functions,rational functions, or transcendental functions such as exponential,logarithmic, hyperbolic functions, power functions, or other periodicfunctions.

Increasing the scan angle sampling may also be performed advantageouslyover selected sections of a scan pattern in order to increase theredundancy of image capture. For example, it may be advantageous tocapture vertical imagery with a higher sample rate than other imagery.This higher sample rate results in an increased redundancy due to thehigher overlap between adjacent frames. The increased redundancy mayallow for an improved vertical product, in particular where the imagequality may vary between captured images. Variable image quality mayoccur due to variable dynamics during capture, specular imagereflections from the area, or other sources.

FIG. 39 a shows a modified set of scan patterns with increased scanangle sampling based on the scan patterns of FIG. 38 a . In particular,the imagery on the straight path scan patterns 113, 114 may have anincreased scan angle sample rate over selected frames 127, 128 towardsthe y-axis where the obliqueness of imagery is smallest (i.e. the imagesare closest to vertical). FIG. 39 b shows a modified set of scanpatterns with increased scan angle sampling around the selected set oflower obliqueness frames 127, 128 based on the scan patterns of FIG. 38b.

FIGS. 38 a, 38 b, 39 a and 39 b give illustrations of scanning camerasystem scan patterns using cropping and increased sampling of scanangles of a scanning mirror to improve the output quality, and in somecases reduce the data storage requirements of an aerial survey. It maybe understood that the geometry of cropping and sampling of scan anglesmay be modified or optimised in a number of ways in order to improve theperformance of the scanning camera system and the quality of generatedimage based products, within the scope of the inventions described inthis specification.

The scanning camera system is suitable for deployment in a wide range ofaerial vehicles for operation over a variety of operating altitudes andground speeds, with a range of GSDs and capture efficiencies.Additionally it is robust to a range of operating conditions such asvariable wind and turbulence conditions that result in dynamicinstabilities such as roll, pitch and yaw of the aerial vehicle. By wayof example, this includes (but is not limited to) twin piston aircraftsuch as a Cessna 310, turboprop aircraft such as a Beechworth KingAir200 and 300 series, and turbofan (jet) aircraft such as a CessnaCitation, allowing aerial imaging from low altitudes to altitudes inexcess of 40,000 feet, at speeds ranging from less than 100 knots toover 500 knots. The aircraft may be unpressurised or pressurised, andeach survey hole may be open or contain an optical glass window asappropriate. Each survey hole may be optionally protected by a doorwhich can be closed when the camera system is not in operation. Othersuitable aerial vehicles include drones, unmanned aerial vehicles (UAV),airships, helicopters, quadcopters, balloons, spacecraft and satellites.

FIG. 40 gives a table that illustrates a range of suitable surveyparameters for the scanning camera system 300 varying from altitude of11,000 ft to 40,000 ft and from ground speed of 240 knots up to groundspeed of 500 knots. The sensors of the cameras of the scanning camerasystem 300 are Gpixel GMAX3265 sensor (9344 by 7000 pixels of pixelpitch 3.2 microns) and the camera lens focal length varies from 300 to900 mm. Each configuration gives a GSD (ground sampling distance) thatis the smallest step between pixels in the captured images. Eachconfiguration is defined according to a flight line spacing, based onwhich a maximum obliqueness (for images used to create verticalorthomosiacs) in degrees and an efficiency in km²/hour may be estimated.The maximum obliqueness is estimated assuming a yaw range of +1-15° andno yaw correction in the stabilisation platform. The table of FIG. 40illustrates a number of features of the scanning camera system 300. TheGSD is seen to decrease with focal length and increase with thealtitude. The maximum obliqueness and efficiency both increase withflight line spacing.

Each configuration of FIG. 40 also includes a timing budget for scandrive units 301, 302, 303. The timing is based on the analysis of scanpatterns such as those shown in FIG. 1 b or 8 b with a required overlapof 10% between adjacent frames. Each scan pattern has a correspondingnumber of frames that increases with focal length due to the smaller GSDand the consequent reduced projection geometry of frames on the ground.

The timing budget in FIG. 40 is the average time available per frame formoving and settling the scanning mirror, latency in the motioncompensation units and the capture and transfer of image data from thecamera to data storage 406. In practice, however, it may be advantageousto allocate a larger time budget for greater angular steps of thescanning mirror, for example when the scan angle resets to start a newscan pattern. Furthermore, the time budget may be eroded by additionalimage captures, for example for the purpose of focus setting. The timingper frame is seen to decrease with GSD in FIG. 40 , that is it decreaseswith focal length and increases with altitude. It also decreases withground speed.

FIG. 41 gives a table that illustrates a range of suitable surveyparameters for the scanning camera system 300 where the sensor of thescanning camera system 300 is an AMS Cmosis CMV50000 CMOS sensor (7920by 6004 pixels of pixel pitch 4.6 microns). The GSD is lower than inFIG. 40 due to the increased pixel pitch, and the timings per frame areconsequently larger. However, the other parameters are essentiallyunchanged. Other suitable sensors include the Vita25k, Python25k, orother RGB, monochrome, multi-spectral, hyperspectral, or infra-redsensors. Different cameras of the scanning camera system may employdifferent sensors. In an alternative embodiment the sensor used in eachscan drive unit may be a monochrome sensor and the overview camera maybe standard RGB. Pan-sharpening using coarse RGB overview pixels and thefine detail monochrome pixels may be used to create high quality colorresolution imagery.

It is noted that the scanning camera system may use an overview camerain order to achieve certain photogrammetry related requirements. Theflight line spacings given in the tables of FIGS. 40 and 41 wereselected based on maximum obliqueness of vertical imagery, and theoverview camera sensor and focal length should be selected such that theprojective geometry 115 of the overview camera is sufficient to achievethose requirements with a given flight line spacing.

The image quality over a survey area may be improved by flying over thearea with a reduced flight line spacing or flying multiple surveys overthe same area. For example, two serpentine flight paths may be flownover a region with flight line orientations that are orthogonal to eachother. This might be achieved by flying with flight lines oriented alongNorth-South directions then East-West directions. Three serpentine pathsmay be flown, for example with relative flight line orientations spacedat 60°. Four serpentine paths may be flown, for example with relativeflight line orientations spaced at 45°. There is a cost in terms of theefficiency of capture when multiple surveys or reduced flight linespacings are used. As can be appreciated by one of skill in the art,additional and/or alternative flight paths can be taken to increase theangular diversity, which may assist with improved 3D meshreconstruction.

In any given scan drive unit, the orientation of a sensor within acamera may be rotated around the optical axis such that the projectiongeometry is modified. Changing the sensor orientation also changes therequirements in terms of mirror geometry, the scan angle steps betweenimage captures, and the flight parameters such as the forward spacingbetween subsequent scan pattern captures.

FIGS. 42 a and 42 b illustrate the updated scan patterns 121, 122 ofscan drive unit 301 when the sensor is rotated by 90° to the portraitsensor orientation. FIGS. 42 c and 42 d illustrate the updated scanpattern 123 of scan drive unit 302 when the sensor is rotated by 90° tothe portrait sensor orientation. FIGS. 42 e and 42 f illustrate theupdated scan pattern 124 of scan drive unit 303 when the sensor isrotated by 90° to the portrait sensor orientation. It is noted that thescan angle steps in the scan patterns 121, 122, 123 124 are smaller thanthe equivalent landscape sensor orientation scan patterns 111, 112, 113,114 respectively.

FIGS. 43 a and 43 b illustrate the calculated mirror geometry of themirror surfaces 314 and/or mirror surface 315 of the scanning mirrorstructure 312 for the portrait sensor orientation. These differ slightlyfrom those for the landscape orientation shown in FIGS. 4 e and 4 f . Itmay be advantageous to use a mirror geometry that is able to handleeither sensor orientation. This may be achieved by using a mirrorgeometry that is the union of the landscape and portrait geometries (forexample the landscape “convex” geometry of FIG. 4 e and the portrait“convex” geometry of FIG. 43 a ). If low reflectivity sections are to beused to allow over-rotation of the mirror without introducing ghostimages then these sections should also be the union of the calculatedsection geometries for the landscape geometry (e.g. “over/dilate” ofFIG. 4 f and “over/dilate” of FIG. 43 b ).

FIG. 43 c illustrates the calculated mirror geometry of the primarymirror 323 of scan drive unit 302 for the portrait sensor orientation.FIG. 43 c also illustrates the calculated geometry of primary mirror 327of scan drive unit 303 for the portrait sensor geometry. These differslightly from those for the landscape sensor orientation illustrated inFIGS. 5 e and 6 e respectively. FIG. 43 d illustrates the calculatedmirror geometry of the secondary mirror 324 of scan drive unit 302 forthe portrait sensor orientation. FIG. 43 c also illustrates thecalculated geometry of secondary mirror 328 of scan drive unit 303 forthe portrait sensor geometry. These differ slightly from those for thelandscape sensor orientation illustrated in FIGS. 5 f and 6 frespectively.

As was the case for the scan drive unit 301, it may be advantageous touse mirror geometries that are able to handle either sensor orientation.This may be achieved by using a mirror geometry that is the union of thelandscape and portrait geometries. For example, scan drive 302 may use aprimary mirror 323 defined by the union of the landscape “convex”geometry of FIG. 5 e and the portrait “convex” geometry of FIG. 43 c .This geometry may also be used for the primary mirror 327 of scan driveunit 303. In the same way, a secondary mirror formed as the union of the“dilate” geometries of FIGS. 5 f and 43 d may be used for the secondarymirror 324 of scan drive unit 302 and also for the secondary mirror 328of scan drive unit 303.

FIGS. 44 a and 44 b show the scan patterns achieved using the scanningcamera system 300 with portrait orientation sensors. The scan patternsinclude curved scan patterns 121, 122 of oblique imagery, and straightscan patterns 123, 124 for the case that the aerial vehicle 110 does notmove between image captures of the scan patterns. FIGS. 44 c and 44 dshow the same scan patterns with the effect of a realistic forwardmotion of the aerial vehicle between image captures. It also showsmultiple scan patterns during a flight line, where the forward spacingbetween scan patterns has been increased relative to the landscapesensor orientation case that was illustrated in FIG. 8 b.

Within the scope of the present disclosure, alternative camera systemsmay be used with a mixture of portrait and landscape sensororientations. For example, a scanning camera system may combine portraitsensor orientation scan drive unit 301 with landscape sensor orientationscan drive units 302, 303, or it may combine landscape sensororientation scan drive unit 301 with portrait sensor orientation scandrive units 302, 303, or other such combinations.

If the vehicle survey aperture is sufficiently large, or if there amultiple apertures in the vehicle, then one or more additional scandrive units may be added to a scanning camera system to improve someaspect of the captured imagery such as quality for 3D reconstruction.One suitable additional scan drive unit 350 is illustrated in FIGS. 45a-45 f . It can be used to capture a single curved scan pattern 130extending from an obliqueness of 22.5° in front of the aerial vehicle110 (on the y-axis) to an obliqueness of 45° to the left of the aerialvehicle 110 (on the x-axis) that is illustrated in FIGS. 45 c and 45 d .Two geometric illustrations of the scan drive unit 350 from differentperspectives are shown in FIG. 45 a and FIG. 45 b . The scan drive unit350 comprises a single sided scanning primary mirror 357 held on anoblique scan axis (elevation θ_(S)=−52.5° and azimuth ϕ_(S)=180°) and afixed secondary mirror 358. The geometric illustration shows theconfiguration with the scan angle of the scan drive 356 set to 0° atwhich angle the primary mirror's 357 surface is oriented with a normaldirected between the z- and y-axes (elevation θ_(M) ¹=−37.5° and azimuthϕ_(M) ¹=0°). The secondary mirror 358 is oriented with a normal opposingthat of the primary mirror 357 when the scan angle is 0° (elevationθ_(M) ¹=52.5° and azimuth ϕ_(M) ¹=180°). There is a single camera 355which is directed downwards at an angle of 7.5° to the vertical z-axis(elevation θ_(S)=−82.5° and azimuth ϕ_(S)=180°).

The scan drive 356 samples scan angles from −32.4° to 0.01° in order togenerate the scan pattern 130. The minimal, dilated, and convex, andsymmetric geometries calculated for the primary mirror 357 are shown inFIG. 45 e along with the axis of rotation and a shifted axis ofrotation. The minimum and dilated geometries of the secondary mirror 358are shown in FIG. 45 f.

Other suitable scan drive units may be designed based on scan drive unit350. For example, scan drive unit 351 is a mirror image of scan driveunit 350 that may be formed by reflecting all components in the y-axisof FIGS. 45 a and 45 b . Scan drive unit 351 generates a single curvedscan pattern 131 extending from an obliqueness of 22.5° in front of theaerial vehicle 110 (on the y-axis) to an obliqueness of 45° to the rightof the aerial vehicle 110 (on the x-axis) that is illustrated in FIGS.46 a and 46 b.

Scan drive unit 352 is a mirror image of scan drive unit 350 that may beformed by reflecting all components in the x-axis of FIGS. 45 a and 45 b. Scan drive unit 352 generates a single curved scan pattern 132extending from an obliqueness of 22.5° behind the aerial vehicle 110 (onthe y-axis) to an obliqueness of 45° to the left of the aerial vehicle110 (on the x-axis) that is illustrated in FIGS. 46 c and 46 d.

Scan drive unit 353 is formed by rotating scan drive unit 350 by 180°around the z-axis of FIGS. 45 a and 45 b . Scan drive unit 353 generatesa single curved scan pattern 133 extending from an obliqueness of 22.5°behind the aerial vehicle 110 (on the y-axis) to an obliqueness of 45°to the right of the aerial vehicle 110 (on the x-axis) that isillustrated in FIGS. 46 a and 46 b.

Scanning camera system 354 comprises the scanning camera system 300 withtwo additional scan drive units 350, 351. The combined scan patterns ofscanning camera system 354 are illustrated in FIGS. 47 a and 47 b .Scanning camera system 355 comprises the scanning camera system 300 withfour additional scan drive units 350, 351, 352, 353. The combined scanpatterns of scanning camera system 354 are illustrated in FIGS. 47 c and47 d.

It may be understood that the scan drive units 350, 351, 352, 353 andscanning camera systems 354, 355 are illustrated in FIGS. 45 a-45 d, 46a-46 d and 47 a-47 d with a portrait sensor orientation, howeveralternative sensor orientations (e.g. landscape) may be used in any ofthe cameras discussed herein within the scope of this specification.

FIGS. 48 a-48 f illustrate scan drive unit 360 which has advantageousproperties in terms of spatial compactness due to the use of a sharedscanning primary mirror 367. Scan drive unit 360 can be used to capturea pair of curved scan patterns 135, 136 each of which start on they-axis and extend left and back relative to the aerial vehicle 110, asshown in FIGS. 48 c and 48 d . Two geometric illustrations of the scandrive unit 360 from different perspectives are shown in FIG. 48 a andFIG. 48 b . The scan drive unit 360 comprises a single sided, sharedscanning primary mirror 367 held on an oblique scan axis (elevationθ_(S)=45° and azimuth ϕ_(S)=0°) and a fixed secondary mirror 368. Thegeometric illustration shows the configuration with the scan angle ofthe scan drive 366 set to 0° at which angle the shared scanning primarymirror's 367 surface is oriented with a normal directed between the z-and y-axes (elevation θ_(M) ¹=−45° and azimuth ϕ_(M) ¹=0°). Thesecondary mirror 368 is oriented with a normal opposing that of theshared scanning primary mirror 367 when the scan angle is 0° (elevationθ_(M) ¹=45° and azimuth ϕ_(M) ¹=180°). There are two cameras 365, 369.The first camera 365 is directed downwards along the vertical z-axis(elevation θ_(S)=)—90° and the second camera 369 is directed downwardsat an angle of 22.5° to the vertical z-axis (elevation θ_(S)=−67.5° andazimuth ϕ_(S)=0°).

Scan drive 366 samples scan angles from −0.01° to 28° in order togenerate the scan patterns 135, 136 simultaneously. The sampling of scanangles may be the same or may be different for each of the cameras 365,369. The minimal, dilated, and convex, and symmetric geometriescalculated for the shared scanning primary mirror 367 are shown in FIG.48 e along with the axis of rotation and a shifted axis of rotation. Theminimum and dilated geometries of the secondary mirror 368 are shown inFIG. 48 f.

Other suitable scan drive units may be designed based on scan drive unit360. For example, scan drive unit 361 is a mirror image of scan driveunit 360 that may be formed by reflecting all components in the y-axisof FIGS. 48 a and 48 b . Scan drive unit 361 generates a pair of curvedscan patterns 137, 138 extending from points on the y-axis backwards andto the right relative to the aerial vehicle 110 as illustrated in FIGS.49 a and 49 b.

Scan drive unit 362 is a mirror image of scan drive unit 360 that may beformed by reflecting all components in the x-axis of FIGS. 48 a and 48 b. Scan drive unit 362 generates a pair of curved scan patterns 139, 140extending from points on the y-axis forwards and to the left relative tothe aerial vehicle 110 as illustrated in FIGS. 49 c and 49 d.

Scan drive unit 363 is formed by rotating scan drive unit 360 by 180°around the z-axis of FIGS. 48 a and 48 b . Scan drive unit 362 generatesa pair of curved scan patterns 141, 142 extending from points on they-axis forwards and to the left relative to the aerial vehicle 110 asillustrated in FIGS. 49 e and 49 f.

FIGS. 50 a to 50 d show a range of perspective views of the combinedcomponents of scan drive units 301, 360, 361 of the scanning camerasystem 364 that were described with respect to FIGS. 4 a-4 f, 48 a-48 fand 49 a-49 f above. Scan drive unit 360 and scan drive unit 361 sit oneither side of the scan drive unit 301 respectively. This arrangement ishighly efficient spatially and advantageous for deployment in a widerange of aerial vehicle camera (survey) holes. FIGS. 50 e and 50 f showthe scan patterns achieved using the scanning camera system 364including curved scan patterns 111, 112 of oblique imagery, and curvedscan patterns 135, 136, 137, 138 of imagery with variable obliqueness.Further to the scan drive unit imaging capability, the scanning camerasystem 364 may additionally include one or more fixed cameras.

FIGS. 51 a-51 f illustrate scan drive unit 370 which has similargeometrical properties to scan drive unit 360 but does not use a sharedscanning mirror. Scan drive unit 370 can be used to capture a singlecurved scan pattern 150 extending from an obliqueness of 22.5° in frontof the aerial vehicle 110 (on the y-axis) back and left relative to theaerial vehicle 110 that is illustrated in FIGS. 51 c and 51 d . Twogeometric illustrations of the scan drive unit 370 from differentperspectives are shown in FIG. 51 a and FIG. 51 b.

The scan drive unit 370 comprises a single sided, scanning primarymirror 377 held on an oblique scan axis (elevation θ_(S)=−45° andazimuth ϕ_(S)=0°) and a fixed secondary mirror 378. The geometricillustration shows the configuration with the scan angle of the scandrive 376 set to 0° at which angle the primary mirror's 377 surface isoriented with a normal directed between the z- and y-axes (elevationθ_(M) ¹=−45° and azimuth ϕ_(M) ¹=0°). The secondary mirror 378 isoriented with a normal opposing that of the primary mirror 377 when thescan angle is 0° (elevation θ_(M) ¹=45° and azimuth ϕ_(M) ¹=180°). Thereis a single camera 375 which is directed downwards at an angle of 22.5°to the vertical z-axis (elevation θ_(S)=−67.5° and azimuth ϕ_(S)=0°).Scan drive 376 samples scan angles from −0.01° to 28° in order togenerate the scan pattern 150. The minimal, dilated, and convex, andsymmetric geometries calculated for the primary mirror 377 are shown inFIG. 51 e along with the axis of rotation and a shifted axis ofrotation. The minimum and dilated geometries of the secondary mirror 378are shown in FIG. 51 f.

Other suitable scan drive units may be designed based on scan drive unit370. For example, scan drive unit 371 is a mirror image of scan driveunit 370 that may be formed by reflecting all components in the y-axisof FIGS. 51 a and 51 b . Scan drive unit 371 generates a single curvedscan pattern 151 extending from an obliqueness of 22.5° in front of theaerial vehicle 110 (on the y-axis) back and to the right of the aerialvehicle 110 that is illustrated in FIGS. 52 a and 52 b.

Scan drive unit 372 is a mirror image of scan drive unit 370 that may beformed by reflecting all components in the x-axis of FIGS. 51 a and 51 b. Scan drive unit 372 generates a single curved scan pattern 152extending from an obliqueness of 22.5° behind the aerial vehicle 110 (onthe y-axis) to an obliqueness of 45° to the left of the aerial vehicle110 (on the x-axis) that is illustrated in FIGS. 52 c and 52 d.

Scan drive unit 373 is formed by rotating scan drive unit 370 by 180°around the z-axis of FIGS. 51 a and 51 b . Scan drive unit 373 generatesa single curved scan pattern 153 extending from an obliqueness of 22.5°behind the aerial vehicle 110 (on the y-axis) to an obliqueness of 45°to the right of the aerial vehicle 110 (on the x-axis) that isillustrated in FIGS. 52 e and 52 f.

Scanning camera system 379 comprises the scan drive units 301, 360, 361,372, 373. The combined scan patterns of scanning camera system 379 areillustrated in FIGS. 53 a and 53 b.

Scanning camera system 381 comprises the scanning camera system 300 withtwo additional scan drive units 372, 373. The combined scan patterns ofscanning camera system 382 are illustrated in FIGS. 53 c and 53 d.

Scanning camera system 382 comprises the scanning camera system 300 withfour additional scan drive units 370, 371, 372, 373. The combined scanpatterns of scanning camera system 382 are illustrated in FIGS. 53 e and53 f.

Scan drive units 301, 302, 303, 350, 351, 352, 353, 360, 361, 362, 363,370, 371, 372, 373 are examples of scan drive units that use a scandrive axis that is parallel to the aerial vehicle of the mirrorsurface(s) that it rotates. Such scan drive units may be referred to astilting scan drive units. Alternative scan drive units may use a scandrive axis that is not parallel to the plane of the mirror surface(s)that it rotates. Such scan drive units employ a spinning mirror and maybe referred to as spinning scan drive units.

FIGS. 54 a-54 f illustrate a spinning scan drive unit 380 with aportrait sensor orientation. The scan drive unit 380 comprises a singlesided scanning primary mirror 383 held on an horizontal scan axis(elevation θ_(S)=−0° and azimuth ϕ_(S)=0°) and a fixed secondary mirror384. The geometric illustration shows the configuration with the scanangle of the scan drive unit 380 set to 0° at which angle the primarymirrors 383 surface is oriented with a normal directed between the z-and y-axes (elevation θ_(M) ¹=−45° and azimuth ϕ_(M) ¹=0°). Thesecondary mirror 378 is oriented with a normal opposing that of theprimary mirror 383 when the scan angle is 0° (elevation θ_(M) ¹=45° andazimuth ϕ_(M) ¹=180°). There is a single camera 376 which is directedvertically downwards (elevation θ_(S)=−90° and azimuth ϕ_(S)=0°). Asshown in FIGS. 54 c and 54 d , scan drive unit 380 generates a singlestraight scan pattern 155 extending from an obliqueness of 45° to theleft of the aerial vehicle (on the x-axis) to an obliqueness of 45° tothe right of the aerial vehicle (on the x-axis) as the scan angle variesbetween −45° and 45°.

Scan drive unit 380 samples scan angles from −45° to 45° in order togenerate the scan pattern 155. In some arrangements, two or more scandrive units 380 may be used, the image captures of the scan pattern 155being split between scan drive units in order to achieve the timingbudget requirements of the system. For example, scan drive unit 380 maysample scan angles from −45° to 0° and a second scan drive unit maysample scan angles and 0° to 45° such that the full range of scan anglesare sampled and the same scan pattern is achieved with roughly doublethe time budge per frame. Scan drive units 302, 303 are used in asimilar way to split a single line scan pattern into two scan patterns113, 114. Any of the scan patterns described in this specification maybe split into parts in the same way, effectively trading off time budgetof image capture against the spatial requirements and additional cost ofthe extra scan drive units.

The minimal, dilated, and convex, and symmetric geometries calculatedfor the primary mirror 383 are shown in FIG. 54 e along with the axis ofrotation and a shifted axis of rotation. The minimum and dilatedgeometries of the secondary mirror 384 are shown in FIG. 54 f.

As can be appreciated by one of skill in the art, any of the scanningcamera systems described herein and obvious variations thereof can beintegrated with one or more of any scan drive unit or scanning camerasystem discussed herein to achieve various timing requirements.Furthermore, the selection of scan angles that define the scan patternsmay be selected according to the requirements and constraints of theoperating conditions such as altitude, flight speed, etc.

As can be appreciated by one of skill in the art, the position of thescan drive in any scan drive unit may be selected at either end of themirror depending on the space available for installation and thegeometry of the scan drive. Furthermore the precise distances betweenmirrors along the optical axis may also be altered in order to achievethe most efficient use of space and minimise occlusions that wouldreduce captured image quality. Small geometric changes such as thesealter the required mirror geometry but do not significantly alter theview directions of captured images. Such changes may allow for more scandrive units to be placed in a constrained space with minimal or noocclusions to give a better imaging system that generates more diverseand/or higher quality captured images.

FIGS. 55 a-55 f illustrate the scan patterns of three scanning camerasystems that employ scan drive unit 380. Scanning camera system 391comprises scan drive units 301, 380. The combined scan patterns ofscanning camera system 391 are illustrated in FIGS. 55 a and 55 b .Scanning camera system 392 comprises the scanning camera system 391 andthe scan drive units 370, 371. The combined scan patterns of scanningcamera system 391 are illustrated in FIGS. 55 c and 55 d . Scanningcamera system 393 comprises the scanning camera system 392 and the scandrive units 372, 373. The combined scan patterns of scanning camerasystem 393 are illustrated in FIGS. 55 e and 55 f.

As shown in FIGS. 56 a and 56 b , scan drive unit 385 is formed byrotating scan drive unit 380 by 45° around the z-axis of FIGS. 54 a and54 b and sampling an extended range of scan angles from −50.4° to 50.4°.Scan drive unit 385 generates a single straight scan pattern 156extending from an obliqueness of 50.4° in front and to the left of theaerial vehicle to an obliqueness of 50.4° behind and to the right of theaerial vehicle.

As shown in FIGS. 56 c and 56 d , scan drive unit 386 is formed byrotating scan drive unit 380 by −45° around the z-axis of FIGS. 54 a and54 b and sampling an extended range of scan angles from −50.4° to 50.4°.Scan drive unit 386 generates a single straight scan pattern 157extending from an obliqueness of 50.4° in front and to the right of theaerial vehicle to an obliqueness of 50.4° behind and to the left of theaerial vehicle.

Scanning camera system 394 comprises the scan drive units 385,386. Thecombined scan patterns of scanning camera system 394 are illustrated inFIGS. 56 e and 56 f . In some arrangements, two or more of scan driveunits 385, 386 may be used, and the image captures of the scan pattern156, 157 being split between scan drive units in order to achieve thetiming budget requirements of the system.

As previously mentioned, any of the scanning camera systems describedherein and obvious variations thereof can be integrated with one or moreof any scan drive unit or scanning camera system discussed herein toachieve various timing requirements.

FIGS. 57 a to 57 e illustrate a number of scan drive units and/orscanning camera systems based on scan drive unit 380, each of whichemploys a camera with a lens of focal length 600 mm and aperture 120 mmfocusing light onto AMS Cmosis CMV50000 CMOS sensor. Scan drive unit 387has the same geometry as scan drive unit 380, but samples a reducedrange of scan angles from −15° to 30.2° to generate the short straightscan pattern 160 shown in FIG. 57 a . Scan drive unit 388 is formed byrotating scan drive unit 380 by 22.5° about the x-axis. Scan drive unit388 samples a reduced range of scan angles from −30.2° to 15° togenerate the short straight scan pattern 161 shown in FIG. 57 b . Scandrive unit 389 is formed by rotating scan drive unit 380 by 22.5° aboutan axis at −30° degrees from the x-axis in the horizontal plane. Scandrive unit 389 samples a reduced range of scan angles from −28° to 47.5°to generate the straight scan pattern 162 shown in FIG. 57 c . Scandrive unit 390 is formed by rotating scan drive unit 380 by 22.5° aboutan axis at 30° degrees from the x-axis in the horizontal plane. Scandrive unit 390 samples a reduced range of scan angles from −47.5° to 28°to generate the straight scan pattern 163 shown in FIG. 57 d.

Scanning camera system 395 comprises scan drive units 387, 378, 389, 390in addition to a modified scan drive unit 301. The modified scan driveunit 301 uses a portrait orientation AMS Cmosis CMV50000 CMOS sensorsand lenses with focal length 600 mm and aperture 120 mm. FIGS. 57 e and57 f illustrate the combined scan patterns of scanning camera system395.

FIGS. 58 a and 58 b show perspective views of a scan drive unit 501 withthree cameras 506, 507, 508 that may be used to capture three scanpatterns 160, 161, 162 with circular arcs centred around an elevation of45°, as shown in FIGS. 58 c and 58 d . The three scan patterns 160, 161,162 combine to form a complete circle, as illustrated in FIGS. 58 c and58 d. Scan drive unit 501 comprises a scanning mirror structure 502attached to a scan drive 503 on a vertical scan axis (elevationθ_(S)=−90° and azimuth ϕ_(S)=0°). In one embodiment, the scanning mirrorstructure 502 is double-sided. The geometric illustration shows theconfiguration with the scan angle of the scan drive 503 set to 0° sothat the first mirror surface 504 is oriented (elevation θ_(S)=0° andazimuth ϕ_(M) ¹=0°) with its normal directed toward the first camera 506along the y-axis. A second mirror surface 505 is mounted on the oppositeside of the scanning mirror structure 502 and directed between thecamera 507 and camera 508.

The cameras 506, 507 and 508 are oriented downward at an oblique anglebut azimuths spaced at 120° (camera 506 elevation θ_(S)=−45°, azimuthϕ_(S)=180°; camera 507 elevation θ_(S)=−45° and azimuth

ϕ_(S)=60°; camera 508 elevation θ_(S)=−45° and azimuth ϕ_(S)=)—60°. Thecameras 506, 507, 508 utilise the Gpixel GMAX3265 sensor (9344 by 7000pixels of pixel pitch 3.2 microns). The camera lenses may have a focallength of 215 mm and aperture of 120 mm (corresponding to F1.8). Thislower focal length generates lower image resolution but a wider scanpattern that may be advantageous in terms of the flight line spacing andefficiency of capture.

FIG. 58 e shows various mirror geometries calculated for the scan driveunit 501. These include the minimum geometry (“min”), a dilated minimumgeometry that is extended by 5 mm beyond the minimum geometry around itsperimeter (“dilate”) and a dilated convex geometry that is the convexhull of the dilated minimum geometry (“convex”). FIG. 58 f shows thedilated convex geometry again (“convex”), and also an extended geometrythat might be required if the range of scan angles is extended by 7.5°at each end of the scan angle range (“over”) to increase the overlapregion between the scan patterns.

Scan drive unit 509 is based on scan drive unit 302, however the camera321 uses a Gpixel GMAX3265 sensor and a lens of focal length 215 mm andaperture of 120 mm (corresponding to F1.8). Further, scan drive 322samples a modified range of scan angles from −10.25° to 10.25° togenerate the straight scan pattern 165 shown in FIGS. 59 a and 59 b .Scanning camera system 510 comprises scan drive units 501, 509 togenerate a combined scan pattern illustrated in FIGS. 59 c and 59 d.

FIGS. 60 a and 60 b show a scan drive unit 511 with four cameras 516,517, 518, 519 from different perspectives that may be used to capturefour scan patterns 170, 171, 172, 173 with circular arcs centred aroundan elevation of 45° that combine to form a complete circle. Top down andoblique views of the scan patterns from the four cameras 516, 517, 518,519 of this scan drive unit 511 are shown in FIGS. 60 c and 60 d . Scandrive unit 511 comprises a scanning mirror structure 512 attached to ascan drive 513 on a vertical scan axis (elevation θ_(S)=−90° and azimuthϕ_(S)=0°). In one embodiment, the scanning mirror structure 512 isdouble-sided. The geometric illustration shows the configuration withthe scan angle of the scan drive set to 0° so that the first mirrorsurface 514 is oriented (elevation θ_(M) ¹=0° and azimuth ϕ_(M) ¹=0°)with its normal directed between camera 516 and camera 517 along they-axis. A second mirror surface 515 is mounted on the opposite side ofthe scanning mirror structure 512 and directed between camera 518 andcamera 519. The cameras 516, 517, 518, 519 are oriented downward at anoblique angle but azimuths spaced at either 60° or 120° to each other(camera 516 elevation θ_(C)=−45°, azimuth ϕ_(C)=150°; camera 517elevation θ_(C)=−45° and azimuth ϕ_(C)=−150°; camera 518 elevationθ_(C)=−45° and azimuth ϕ_(C)=−30°; camera 519 elevation θ_(C)=−45° andazimuth ϕ_(C)=30°).

Each camera 516, 517, 518, 519 samples the scan angles of the scan drive513 over a range of 45° in order to achieve a one quarter circle scanpattern arc. The uneven azimuthal spacing of the cameras 516, 517, 518,519 around the scanning mirror structure 512 may be advantageous interms of the timing budget of capture and the simultaneous use of thescanning mirror structure 512 to capture images on the cameras 516, 517,518, 519. Scan drive 511 generates the same scan pattern that would beachieved with scan drive unit 301 sampling scan angles in the range −45°to 45°. The use of additional cameras may be advantageous as it reducesthe size of scanning mirror structure 512 required to achieve thecapture. This arrangement may also be advantageous in terms ofrobustness of yaw of the aerial vehicle 110 as the scan pattern capturesa full 360° range in azimuth.

FIG. 60 e shows various mirror geometries calculated for the scan driveunit 511. These include the minimum geometry (“min”), a dilated minimumgeometry that is extended by 5 mm beyond the minimum geometry around itsperimeter (“dilate”) and a dilated convex geometry that is the convexhull of the dilated minimum geometry (“convex”). FIG. 60 f shows thedilated convex geometry again (“convex”), and also an extended geometrythat might be required if the range of scan angles is extended by 7.5°at each end of the scan angle range (“over”) to increase the overlapregion between the scan patterns.

FIGS. 61 a and 61 b show perspective views of a scan drive unit 521 withfour cameras 526, 527, 528, 529 that may be used to capture four scanpatterns 175, 176, 177, 178 with circular arcs, as shown in FIGS. 61 cand 61 d ). Top down and oblique views of the scan patterns 175, 176,177, 178 from the four cameras 526, 527, 528, 529 of scan drive unit 521are shown in FIGS. 61 c and 61 d.

Scan drive unit 521 comprises a scanning mirror structure 522 attachedto a scan drive 523 on a vertical scan axis (elevation θ_(S)=−90° andazimuth ϕ_(S)=0°). In one embodiment, the scanning mirror structure 522is double-sided. The geometric illustration in FIGS. 61 a and 61 b showthe configuration with the scan angle of the scan drive 523 set to 0° sothat the first mirror surface 524 is oriented (elevation θ_(M) ¹=0° andazimuth ϕ_(M) ¹=0°) with its normal directed between camera 526 andcamera 527 along the y-axis. A second mirror surface 525 is mounted onthe opposite side of the scanning mirror structure 522 and directedbetween camera 528 and camera 529. The cameras 526, 527, 528, 529 areoriented downward at an oblique angle and azimuthally spaced 90° to eachother (camera 526 elevation θ_(C)=−47°, azimuth ϕ_(C)=135°; camera 527elevation θ_(C)=−43° and azimuth ϕ_(C)=45°; camera 528 elevationθ_(S)=−47° and azimuth ϕ_(C)=−45°; camera 529 elevation θ_(C)=−43° andazimuth ϕ_(C)=−43°).

Each camera 526, 527, 528, 529 samples the scan angles of the scan drive523 over a range of 60° in order to achieve a one third circle scanpattern arc. The use of two different elevations of cameras 529, 527compared to cameras 526, 528 directed at the shared scanning mirrorstructure 522 means that the arcs do not overlap and capturecomplementary regions of the object area to the sides of the aerialvehicle 110. This may be advantageous in terms of the efficiency of thescanning camera system as a larger flight line spacing may be used whilemaintaining some required distribution of oblique image captures to theleft and right sides of the aerial vehicle 110. It may also beadvantageous in improving the quality of image capture for obliqueimagery and the generation of a 3D model. This arrangement may also beadvantageous in terms of robustness of yaw of the aerial vehicle 110 asthe scan pattern captures a full 360° range in azimuth.

FIG. 61 e shows various mirror geometries calculated for the scan driveunit 521. These include the minimum geometry (“min”), a dilated minimumgeometry that is extended by 5 mm beyond the minimum geometry around itsperimeter (“dilate”) and a dilated convex geometry that is the convexhull of the dilated minimum geometry (“convex”). FIG. 61 f shows thedilated convex geometry again (“convex”), and also an extended geometrythat might be required if the range of scan angles is extended by 7.5°at each end of the scan angle range (“over”) to increase the overlapregion between the scan patterns.

Scan drive unit 530 has the same geometry as scan drive unit 302, butsamples a modified range of scan angles from −10.25° to 10.25° togenerate the short straight scan pattern 179 shown in FIGS. 62 and 62 b.Scan pattern 179 may be used to generate high quality vertical imagecaptures. Scanning camera system 531 comprises scan drive units 530, 511to generate the combined scan pattern shown in FIGS. 62 c and 62 d .Scanning camera system 532 comprises scan drive units 530, 521 togenerate the combined scan pattern shown in FIGS. 62 e and 62 f.

Scan drive unit 535 has the same geometry as scan drive unit 380, butsamples a reduced range of scan angles from −22.5° to 22.5° to generatethe short straight scan pattern 180 shown in FIGS. 63 a and 63 b . Scanpattern 180 may be used to generate high quality vertical imagecaptures. Scanning camera system 536 comprises scan drive units 535 andscan drive unit 511 to generate the combined scan pattern shown in FIGS.63 c and 63 d . Scanning camera system 537 comprises scan drive units535, 521 to generate the combined scan pattern shown in FIGS. 63 e and63 f.

Obviously, numerous modifications and variations are possible in lightof the above teachings. It is therefore to be understood that within thescope of the appended claims, the invention may be practiced otherwisethan as specifically described herein.

Thus, the foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. As will be understood by thoseskilled in the art, the present invention may be embodied in otherspecific forms without departing from the spirit or essentialcharacteristics thereof. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting of the scopeof the invention, as well as other claims. The disclosure, including anyreadily discernible variants of the teachings herein, defines, in part,the scope of the foregoing claim terminology such that no inventivesubject matter is dedicated to the public.

Embodiments of the present disclosure may also be as set forth in thefollowing parentheticals.

(1) An imaging system, comprising: a camera configured to capture animage on an object area from an imaging beam from the object area, thecamera including an image sensor and a lens; one or more glass platespositioned between the image sensor and the lens of the camera; one ormore first drives coupled to each of the one or more glass plates; ascanning mirror structure including at least one mirror surface; asecond drive coupled to the scanning mirror structure and configured torotate the scanning mirror structure about a scan axis based on a scanangle; and a motion compensation system configured to determine at leastone of plate rotation rates and plate rotation angles based on relativedynamics of the imaging system and the object area and opticalproperties of the one or more glass plates; and control the one or morefirst drives to rotate the one or more glass plates about one or morepredetermined axes based on at least one of corresponding plate rotationrates and plate rotation angles.

(2) The system of (1), wherein the image sensor is exposed to theimaging beam synchronously with movement of the one or more glassplates.

(3) The system of any (1) to (2), wherein the motion compensation systemis configured to continuously move the one or more glass plates duringcapture of images by the camera.

(4) The system of any (1) to (3), wherein a scan axis of the one or morefirst drives is selected from one of: substantially perpendicular to anoptical axis of the camera; and substantially parallel to the opticalaxis of the camera.

(5) The system of any (1) to (4), wherein the motion compensation systemis configured to obtain a region of interest in each of captured imagesand estimate pixel velocity using the regions of interest.

(6) The system of any (1) to (5), wherein the motion compensation systemis configured to estimate at least one of motion pixel velocity andattitude rate pixel velocity; and control the one or more first drivesbased upon one of the motion pixel velocity and the attitude rate pixelvelocity.

(7) The system of any (1) to (6), wherein the attitude rate pixelvelocity is a yaw rate pixel velocity.

(8) The system of any (1) to (7), wherein the motion pixel velocity is aforward motion pixel velocity.

(9) The system of any (1) to (8), wherein the motion compensation systemis configured to control the one or more first drives based upon asleast one of: motion of the imaging system relative to the object area;scan angle; projection geometry; alignment of the one or more glassplates; characteristics of the one or more glass plates; opticalproperties of the one of more glass plates; alignment of the imagingsystem relative to a flight path; and a rate of change of attitude ofthe imaging system relative to the object area.

(10) An imaging method, comprising: reflecting an imaging beam from anobject area using at least one mirror surface of a scanning mirrorstructure to an image sensor of a camera to capture a set of imagesalong a scan path of the object area, the camera comprising a lens andan image sensor; capturing an image from the imaging beam from theobject area reflected by the at least one mirror surface using the imagesensor of the camera; positioning one or more glass plates between theimage sensor and the lens of the camera; determining plate rotationrates and plate rotation angles based on one of characteristics of thecamera, characteristics and positioning of the one or more glass plates,and relative dynamics of the camera and the object area; and rotatingthe one or more glass plates about one or more predetermined axes basedon corresponding plate rotation rates and plate rotation angles.

(11) The method of (10), wherein the image sensor is exposed to theimaging beam synchronously with movement of the one or more glassplates.

(12) The method of any (10) to (11), comprising continuously moving theone or more glass plates during capture of images by the camera.

(13) The method of any (10) to (12), comprising: obtaining a region ofinterest in each of captured images; and estimating pixel velocity usingthe regions of interest.

(14) The method of any (10) to (13), comprising: estimating at least oneof motion pixel velocity and attitude rate pixel velocity; andcontrolling the one or more first drives based upon one of the motionpixel velocity and the attitude rate pixel velocity.

(15) The method of any (10) to (14), comprising determining at least oneof the plate rotation rates and plate rotation angles based upon atleast one of: motion of the camera relative to the object area; scanangle; projection geometry; alignment of the one or more glass plates;characteristics of the one or more glass plates; optical properties ofthe one of more glass plates; alignment relative to a flight path; and arate of change of attitude of the camera relative to the object area.

LISTING OF REFERENCE NUMERALS

-   110: aerial vehicle-   111: scan pattern-   112: scan pattern-   113: scan pattern-   114: scan pattern-   115: projective geometry-   116: grid line-   117: grid line-   118: grid line-   119: grid line-   121: scan pattern-   122: scan pattern-   123: scan pattern-   124: scan pattern-   125: frame-   126: frame-   127: frames-   128: frames-   130: scan pattern-   131: scan pattern-   132: scan pattern-   133: scan pattern-   135: scan pattern-   136: scan pattern-   137: scan pattern-   138: scan pattern-   139: scan pattern-   140: scan pattern-   141: scan pattern-   142: scan pattern-   150: scan pattern-   151: scan pattern-   152: scan pattern-   153: scan pattern-   155: scan pattern-   156: scan pattern-   157: scan pattern-   160: scan pattern-   161: scan pattern-   162: scan pattern-   163: scan pattern-   165: scan pattern-   170: scan pattern-   171: scan pattern-   172: scan pattern-   173: scan pattern-   175: scan pattern-   176: scan pattern-   177: scan pattern-   178: scan pattern-   179: scan pattern-   180: scan pattern-   210: flight line-   211: flight line-   212: flight line-   213: flight line-   214: flight line-   215: flight line-   220: turning path-   221: turning path-   222: turning path-   223: turning path-   224: turning path-   225: turning path-   226: flight line spacing-   230: viewing direction-   231: viewing direction-   232: viewing direction-   233: viewing direction-   234: viewing direction-   235: viewing direction-   236: circle of viewing directions at fixed elevations-   237: circle of viewing directions at fixed elevations-   238: circle of viewing directions at fixed elevations-   240: lens-   241: optical plate-   242: optical plate-   243: sensor-   244: optical plate-   245: optical plate-   246: optical plate-   250: camera-   251: area-   252: mirror-   253: survey hole-   254: location-   255: beam-   256: mirror-   257: beam-   258: location-   259: beam-   260: low-reflective material-   261: beam-   262: beam-   263: beam-   265: low-reflective material-   266: mirror-   271: sensor location-   272: sensor location-   273: sensor location-   274: sensor location-   275: sensor location-   276: sensor location-   277: sensor location-   278: sensor location-   279: sensor location-   281: rotation-   282: rotation-   283: rotation-   284: rotation-   290: light ray-   291: optical plate-   292: front surface-   293: rear surface-   294: curved path of viewing directions in the hemisphere-   295: curved path of viewing directions in the hemisphere-   296: curved path of viewing directions in the hemisphere-   297: curved path of viewing directions in the hemisphere-   300: scanning camera system-   301: scan drive unit-   302: scan drive unit-   303: scan drive unit-   305: survey hole-   310: camera-   311: camera-   312: scanning mirror structure-   313: scan drive-   314: mirror surface-   315: mirror surface-   316: axis of rotation-   317: low-reflective material-   321: camera-   322: scan drive-   323: primary mirror-   324: secondary mirror-   325: camera-   326: scan drive-   327: primary mirror-   328: secondary mirror-   350: scan drive unit-   351: scan drive unit-   352: scan drive unit-   353: scan drive unit-   354: scanning camera system-   355: scanning camera system-   356: scan drive-   357: primary mirror-   358: secondary mirror-   360: scan drive unit-   361: scan drive unit-   362: scan drive unit-   363: scan drive unit-   364: scanning camera system-   365: camera-   366: scan drive-   367: primary mirror-   368: secondary mirror-   369: camera-   370: scan drive unit-   371: scan drive unit-   372: scan drive unit-   373: scan drive unit-   375: camera-   376: scan drive-   377: primary mirror-   378: secondary mirror-   379: scanning camera system-   380: scan drive unit-   381: scanning camera system-   382: scanning camera system-   383: primary mirror-   384: secondary mirror-   385: scan drive unit-   386: scan drive unit-   387: scan drive unit-   388: scan drive unit-   389: scan drive unit-   390: scan drive unit-   391: scanning camera system-   392: scanning camera system-   393: scanning camera system-   394: scanning camera system-   395: scanning camera system-   401: auto-pilot-   402: pilot display-   403: pilot input-   404: GNSS receiver-   405: system control-   406: data storage-   407: stabilisation platform-   408: scanning camera system-   409: IMU-   410: camera (s)-   411: scan drive unit-   412: scan drive unit-   413: scanning mirror-   414: camera-   415: motion compensation unit-   416: camera-   417: motion compensation unit-   430: scan angle-   431: mirror drive-   432: mirror control-   433: mirror sensor-   434: scan drive unit parameters-   435: motion compensation data-   436: IMU attitude data-   437: mirror control data-   438: focus data-   439: pixel data-   440: ROI pixel velocity estimator-   450: geometry estimator module-   451: projection geometry-   452: forward motion pixel velocity estimator-   453: forward motion pixel velocity-   454: attitude rate pixel velocity estimator-   455: attitude rate pixel velocity-   456: attitude rate pixel velocity estimator-   457: ROI pixel velocity-   458: motion compensation control-   459: motion compensation sensor-   460: motion compensation drive-   461: motion compensation calibration data-   462: ground velocity-   463: latitude/longitude data-   464: altitude data-   465: DEM data-   466: IMU attitude data-   467: SDU geometry data-   468: IMU attitude rates-   469: ROI images-   470: gimbal angles-   471: survey hole geometry-   472: mirror data-   473: vignetting data-   474: vignetting analysis process-   475: processing step-   476: refinement step-   477: camera poses, positions and additional data-   478: 3D surface reconstruction-   479: orthomosaic generation-   480: vignetting compensation-   482: orthomosaic-   501: scan drive unit-   502: scanning mirror structure-   503: scan drive-   504: mirror surface-   505: mirror surface-   506: camera-   507: camera-   508: camera-   509: scan drive unit-   510: scanning camera system-   511: scan drive unit-   512: scanning mirror structure-   513: scan drive-   514: mirror surface-   515: mirror surface-   516: camera-   517: camera-   518: camera-   519: camera-   521: scan drive unit-   522: scanning mirror structure-   523: scan drive-   524: mirror surface-   525: mirror surface-   526: camera-   527: camera-   528: camera-   529: camera-   530: scan drive unit-   531: scanning camera system-   532: scanning camera system-   535: scan drive unit-   536: scanning camera system-   537: scanning camera system

1. An imaging system, comprising: a camera configured to capture animage of an object area from an imaging beam from the object area, thecamera including an image sensor and a lens; one or more glass platespositioned between the image sensor and the lens of the camera; one ormore first drives coupled to each of the one or more glass plates; ascanning mirror structure including at least one mirror surface; asecond drive coupled to the scanning mirror structure and configured torotate the scanning mirror structure about a scan axis based on a scanangle; and a motion compensation system configured to determine at leastone of plate rotation rates and plate rotation angles based on relativedynamics of the imaging system and the object area and opticalproperties of the one or more glass plates; and control the one or morefirst drives to rotate the one or more glass plates about one or morepredetermined axes based on at least one of corresponding plate rotationrates and plate rotation angles.
 2. The imaging system according toclaim 1, wherein the image sensor is exposed to the imaging beamsynchronously with movement of the one or more glass plates.
 3. Theimaging system according to claim 1, wherein the motion compensationsystem is configured to continuously move the one or more glass platesduring capture of images by the camera.
 4. The imaging system accordingto claim 1, wherein a scan axis of the one or more first drives isselected from one of: substantially perpendicular to an optical axis ofthe camera; and substantially parallel to the optical axis of thecamera.
 5. The imaging system according to claim 1, wherein the motioncompensation system is configured to obtain a region of interest in eachof captured images and estimate pixel velocity using the regions ofinterest.
 6. The imaging system according to claim 1, wherein the motioncompensation system is configured to estimate at least one of motionpixel velocity and attitude rate pixel velocity; and control the one ormore first drives based upon one of the motion pixel velocity and theattitude rate pixel velocity.
 7. The imaging system according to claim6, wherein the attitude rate pixel velocity is a yaw rate pixelvelocity.
 8. The imaging system according to claim 6, wherein the motionpixel velocity is a forward motion pixel velocity.
 9. The imaging systemaccording to claim 1, wherein the motion compensation system isconfigured to control the one or more first drives based upon as leastone of: motion of the imaging system relative to the object area; scanangle; projection geometry; alignment of the one or more glass plates;characteristics of the one or more glass plates; optical properties ofthe one of more glass plates; alignment of the imaging system relativeto a flight path; and a rate of change of attitude of the imaging systemrelative to the object area.
 10. An imaging method, comprising:reflecting an imaging beam from an object area using at least one mirrorsurface of a scanning mirror structure to an image sensor of a camera tocapture a set of images along a scan path of the object area, the cameracomprising a lens and an image sensor; capturing an image from theimaging beam from the object area reflected by the at least one mirrorsurface using the image sensor of the camera; positioning one or moreglass plates between the image sensor and the lens of the camera;determining plate rotation rates and plate rotation angles based on oneof characteristics of the camera, characteristics and positioning of theone or more glass plates, and relative dynamics of the camera and theobject area; and rotating the one or more glass plates about one or morepredetermined axes based on corresponding plate rotation rates and platerotation angles.
 11. The imaging method according to claim 10, whereinthe image sensor is exposed to the imaging beam synchronously withmovement of the one or more glass plates.
 12. The imaging methodaccording to claim 10, comprising continuously moving the one or moreglass plates during capture of images by the camera.
 13. The imagingmethod according to claim 10, comprising: obtaining a region of interestin each of captured images; and estimating pixel velocity using theregions of interest.
 14. The imaging method according to claim 10,comprising: estimating at least one of motion pixel velocity andattitude rate pixel velocity; and controlling the one or more firstdrives based upon one of the motion pixel velocity and the attitude ratepixel velocity.
 15. The imaging method according to claim 10, comprisingdetermining at least one of the plate rotation rates and plate rotationangles based upon at least one of: motion of the camera relative to theobject area; scan angle; projection geometry; alignment of the one ormore glass plates; characteristics of the one or more glass plates;optical properties of the one of more glass plates; alignment relativeto a flight path; and a rate of change of attitude of the camerarelative to the object area.